Image detection method and related equipment

Abstract

The invention provides an image detection method and related equipment. The method comprises the following steps: acquiring a to-be-detected target image; extracting preset various types of image features from the target image, and performing image detection on the target image based on the extracted various types of image features to obtain a first detection result; and extracting image features from a plurality of image areas included in the target image, and performing image detection on the target image based on the extracted image features corresponding to the plurality of image areas to obtain a second detection result, and in response to at least one of the first detection result and the second detection result indicating that the target image does not pass the image detection, determining that the target image does not pass the image detection.

Description

Technical Field

[0001] One or more embodiments of the present specification relate to the field of image detection technology, and in particular, to an image detection method and related equipment. Background Art

[0002] Generative AI (AI-Generated Content, AIGC) is a type of AI technology that can generate various types of content based on given input or instructions, such as text, images, audio and video, bringing convenience to people's lives and work. However, various AI images generated by AIGC technology, such as face images and ID images, are sometimes maliciously used in the identity authentication process of various applications in an attempt to impersonate real users for identity authentication, seriously endangering the interests of users and network security.

[0003] Moreover, with the rapid development of AIGC technology, facial images and ID images generated by AIGC technology are more difficult to be recognized. Therefore, how to improve the accuracy of image detection is an urgent problem to be solved. Summary of the invention

[0004] In view of this, one or more embodiments of the present specification provide an image detection method and related devices.

[0005] In a first aspect, this specification provides an image detection method, the method comprising:

[0006] Acquire the target image to be detected;

[0007] Extracting preset multiple types of image features from the target image respectively, and performing image detection on the target image based on the extracted multiple types of image features to obtain a first detection result; and,

[0008] Extracting image features from a plurality of image regions included in the target image respectively, and performing image detection on the target image based on the extracted image features corresponding to the plurality of image regions to obtain a second detection result;

[0009] In response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection, it is determined that the target image fails the image detection.

[0010] In a second aspect, this specification provides an image detection device, the device comprising:

[0011] An acquisition unit, used for acquiring a target image to be detected;

[0012] a multi-type feature extraction unit, configured to extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result;

[0013] a multi-region feature extraction unit, configured to extract image features from a plurality of image regions included in the target image, respectively, and perform image detection on the target image based on the extracted image features corresponding to the plurality of image regions, so as to obtain a second detection result;

[0014] An image detection unit is used to determine that the target image fails the image detection in response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection.

[0015] Correspondingly, this specification also provides an electronic device, comprising: a memory and a processor; the memory stores a computer program / instruction that can be executed by the processor; when the processor executes the computer program / instruction, it executes the image detection method described in the first aspect above.

[0016] Accordingly, the present specification also provides a computer-readable storage medium on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the image detection method as described in the first aspect above is executed.

[0017] Accordingly, the present specification also provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the image detection method described in the first aspect above is executed.

[0018] In summary, the present application can first obtain a target image to be detected, and then extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result. In addition, image features can also be extracted from multiple image areas contained in the target image, and image detection can be performed on the target image based on the extracted image features corresponding to the multiple image areas to obtain a second detection result. Further, in response to at least one of the first detection result and the second detection result indicating that the target image has not passed the image detection, it can be determined that the target image has not passed the image detection. In this way, the present application performs image detection on the target image to be detected by two different image detection methods, one of which is based on multiple different types of image features extracted from the target image, and the other is based on image features extracted from multiple different image areas of the target image. Combined with the detection results of the above two image detection methods, as long as at least one of the detection results indicates that the target image has not passed the image detection, it can be determined that the target image has not passed the image detection, thereby greatly improving the strength and accuracy of image detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 is a schematic diagram of a system architecture provided by an exemplary embodiment;

[0020] Figure 2 is a flowchart of an image detection method provided by an exemplary embodiment;

[0021] Figure 3 It is a schematic diagram of an image detection process based on multi-type image feature extraction provided by an exemplary embodiment;

[0022] Figure 4 It is a schematic diagram of an image detection process based on multi-region image feature extraction provided by an exemplary embodiment;

[0023] Figure 5 is a schematic diagram of knowledge distillation provided by an exemplary embodiment;

[0024] Figure 6 is a structural schematic diagram of an image detection device provided by an exemplary embodiment;

[0025] Figure 7 It is a schematic structural diagram of an electronic device provided by an exemplary embodiment. DETAILED DESCRIPTION

[0026] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with one or more embodiments of this specification. Instead, they are merely examples of devices and methods consistent with some aspects of one or more embodiments of this specification as detailed in the appended claims.

[0027] It should be noted that: in other embodiments, the steps of the corresponding method are not necessarily performed in the order shown and described in this specification. In some other embodiments, the steps included in the method may be more or less than those described in this specification. In addition, a single step described in this specification may be decomposed into multiple steps for description in other embodiments; and multiple steps described in this specification may be combined into a single step for description in other embodiments.

[0028] It should be noted that the “plurality” mentioned in this application refers to two or more than two.

[0029] In addition, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0030] First, some terms in this specification are explained to facilitate understanding by those skilled in the art.

[0031] (1) Electronic Know Your Customer (eKYC) is a technology that completes user identity verification electronically. eKYC can quickly and effectively identify all kinds of identity documents around the world, automatically extract key identity information and complete identity verification. It can also input the extracted identity information into a database or corresponding system, thereby replacing the cumbersome manual entry process. In addition, eKYC can also perform biometric verification, such as real-time facial image acquisition through a camera, detecting and identifying facial expressions and movements during the interaction process, so as to determine whether the user is a real person, effectively prevent identity fraud, ensure the authenticity and reliability of the user, etc., which will not be elaborated here.

[0032] As mentioned above, with the rapid development of AIGC technology, face images and ID images generated by AIGC may be maliciously used in the process of electronic identity authentication, seriously endangering the interests of users and network security. Among them, the ID images generated by AIGC may include ID images of forged IDs (such as forged personal identification documents, driver's licenses and passports, etc.) generated by AIGC technology, etc., and this specification does not make specific limitations on this.

[0033] Based on this, this specification provides a technical solution that can detect images simultaneously through two different image detection methods, thereby improving the image detection strength and accuracy.

[0034] In implementation, the target image to be detected can be first acquired, and then multiple types of preset image features can be extracted from the target image, and image detection can be performed on the target image based on the extracted multiple types of image features to obtain a first detection result. In addition, image features can be extracted from multiple image regions contained in the target image, and image detection can be performed on the target image based on the extracted image features corresponding to the multiple image regions to obtain a second detection result. Furthermore, in response to at least one of the first detection result and the second detection result indicating that the target image has not passed the image detection, it can be determined that the target image has not passed the image detection.

[0035] In the above technical solution, the present application performs image detection on the target image to be detected by two different image detection methods, one of which is based on multiple different types of image features extracted from the target image, and the other is based on image features extracted from multiple different image areas of the target image. In this way, by combining the detection results of the above two image detection methods, as long as at least one of the detection results indicates that the target image has not passed the image detection, it can be determined that the target image has not passed the image detection, thereby greatly improving the strength and accuracy of image detection.

[0036] Exemplarily, taking the target image as an ID card image containing a user ID, through the image detection method provided by the present application, the ID card image generated by the AIGC technology may be able to bypass one of the image detection methods, but as long as another image detection method detects that the user ID card in the ID card image is a forged ID card, it can be determined to be a forged ID card, thereby effectively intercepting the AIGC forged ID card (i.e., fake ID card) that appears in identity verification, thereby safeguarding user interests and network security.

[0037] See also Figure 1 , Figure 1 FIG. 1 is a schematic diagram of a system architecture provided by an exemplary embodiment. Figure 1As shown, the system may include an electronic device 100, an electronic device 200a, an electronic device 200b, and an electronic device 200c. Among them, the electronic device 200a, the electronic device 200b, and the electronic device 200c may respectively run a corresponding client with an identity verification function, such as an eKYC client, or may also be any other possible client with an eKYC function, such as a bank client and a medical service client, etc., which is not specifically limited in this specification. The electronic device 200 may run a server corresponding to the client with an identity verification function, such as an eKYC server, and also a bank server and a medical service client, etc., which is not specifically limited in this specification. In an embodiment shown, a communication connection may be established between the electronic device 100 and the electronic devices 200a, 200b, and 200c in any possible manner, which is not specifically limited in this specification.

[0038] Below, the image detection method provided in the present application will be described by taking the electronic device 100 and the electronic device 200a as examples.

[0039] In one illustrated embodiment, a user may upload a target image to be detected to a client of electronic device 200a. Further, the client in electronic device 200a may send the target image to electronic device 100, specifically, to a server in electronic device 100.

[0040] The target image to be detected may be a document image containing a user's document taken by the user through the electronic device 200a, or may be a face image containing the user's face, etc., which is not specifically limited in this specification. For example, the user's document may be a personal identification document, a driver's license, and a passport of the user, etc., which is not specifically limited in this specification.

[0041] In an illustrated embodiment, after receiving the target image sent by the client, the server in the electronic device 100 can extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result. For details, please refer to the following Figure 2 The description of the corresponding embodiment will not be elaborated here. The first detection result can be used to indicate whether the target image passes or fails the image detection.

[0042] In an illustrated embodiment, the server in the electronic device 100 may further extract image features from the multiple image regions included in the target image, and perform image detection on the target image based on the extracted image features corresponding to the multiple image regions to obtain a second detection result. For details, refer to the following Figure 2The description of the corresponding embodiment will not be elaborated here. The second detection result can be used to indicate whether the target image passes or fails the image detection.

[0043] Further, the server may determine a final detection result based on the first detection result and the second detection result. In an illustrated embodiment, in response to at least one of the first detection result and the second detection result indicating that the target image has not passed the image detection, the server may determine that the final detection result is that the target image has not passed the image detection.

[0044] In an illustrated embodiment, taking the target image as an ID image containing a user ID as an example, performing image detection on the target image may include performing anti-counterfeiting detection on the user ID contained in the target image; accordingly, the above-mentioned first detection result and the second detection result may be used to indicate whether the user ID contained in the target image is a forged ID; accordingly, in response to at least one of the above-mentioned first detection result and the second detection result indicating that the user ID contained in the target image is a forged ID, the server may determine that the final detection result is that the user ID contained in the target image is a forged ID.

[0045] In an illustrated embodiment, taking the target image as a facial image containing a user's face as an example, performing image detection on the target image may include performing facial anti-counterfeiting detection on the target image; accordingly, the above-mentioned first detection result and the second detection result may be used to indicate that the target image is a forged facial image; accordingly, in response to at least one of the above-mentioned first detection result and the second detection result indicating that the target image is a forged facial image, the server may determine that the final detection result is that the target image is a forged facial image.

[0046] Furthermore, the server can send the determined final test result to the client in the electronic device 200a. Accordingly, the client can receive the final test result and display corresponding prompt information to the user based on the final test result, such as outputting a prompt message "Document test failed" or "Document test passed" to the user, etc. This manual does not make any specific limitations on this.

[0047] In an illustrated embodiment, the server may also directly send the first detection result and the second detection result to the electronic device 200a. Accordingly, the client may determine the final detection result based on the received first detection result and the second detection result. For example, in response to at least one of the received first detection result and the second detection result indicating that the user certificate included in the target image is a forged certificate, the client may determine that the user certificate included in the target image is a forged certificate.

[0048] It should be noted that, in some possible implementations, the above-mentioned method steps of image detection can also be performed by the client. Exemplarily, after acquiring the target image to be detected, the client in the electronic device 200a can directly extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result, and extract image features from multiple image areas contained in the target image, and perform image detection on the target image based on the extracted image features corresponding to the multiple image areas to obtain a second detection result. Furthermore, the client can determine the final detection result based on the above-mentioned first detection result and the second detection result, and so on, which will not be repeated here.

[0049] Alternatively, in some possible implementations, the client and the server may respectively perform one of the two image detection methods. Exemplarily, the client may extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result; while the server may extract image features from multiple image regions contained in the target image, and perform image detection on the target image based on the extracted image features corresponding to the multiple image regions to obtain a second detection result, and so on, which is not specifically limited in this specification.

[0050] So far, the present application has simultaneously detected the target image to be detected through two different image detection methods, thereby improving the image detection strength and accuracy. In addition, it should be emphasized that the present application only requires one target image when performing image detection, and there is no need to collect multiple images for repeated detection, thereby improving detection efficiency and cost. Taking the anti-counterfeiting detection of certificate images as an example, the certificate image generated by AIGC technology may be able to bypass one of the image detection methods, but as long as another image detection method detects that the user certificate in the certificate image is a forged certificate, it can be determined to be a forged certificate, thereby effectively intercepting AIGC forged certificates that appear in identity verification and safeguarding user interests and network security.

[0051] In one illustrated embodiment, electronic device 200a, electronic device 200b and electronic device 200c may be smart wearable devices, smart phones, tablet computers, laptop computers and desktop computers, etc. having the above functions, and this specification does not make specific limitations on this; electronic device 100 may be a desktop computer, a server, or a server cluster or a cloud computing center composed of multiple servers, etc. having the above functions, and this specification does not make specific limitations on this.

[0052] See also Figure 2 , Figure 2is a flowchart of an image detection method provided by an exemplary embodiment. The method can be applied to Figure 1 The system architecture shown can be applied to Figure 1 In any electronic device shown. Figure 2 As shown, the method may specifically include the following steps S201 to S204.

[0053] Step S201, obtaining a target image to be detected.

[0054] First, a target image to be detected is obtained. The target image may be a document image containing a user's document, or a face image containing a user's face, etc. This specification does not specifically limit this. In an embodiment shown in FIG. Figure 1 The description of the corresponding embodiments will not be repeated here.

[0055] Step S202: extracting preset multiple types of image features from the target image respectively, and performing image detection on the target image based on the extracted multiple types of image features to obtain a first detection result.

[0056] Furthermore, after acquiring the target image to be detected, two different image detection methods can be used to perform image detection on the target image respectively, and corresponding detection results can be obtained.

[0057] In an illustrated embodiment, two different image detection methods may include: image detection based on multi-type image feature extraction, and image detection based on multi-region image feature extraction.

[0058] Below, image detection based on multi-type image feature extraction is first explained.

[0059] In an illustrated embodiment, based on a preset multiple types of image feature extraction method, preset multiple types of image features can be respectively extracted from the target image, and image detection can be further performed on the target image based on the extracted multiple types of image features to obtain a first detection result.

[0060] It should be noted that the present specification does not specifically limit the above-mentioned multiple types of image features. In an illustrated embodiment, the multiple types of image features may include any of differential features of the target image, frequency domain features, and enhanced features based on Exact Feature Distribution Mixing (EFDMix).

[0061] In an illustrated embodiment, when performing image detection on a target image based on multiple types of extracted image features, the multiple types of extracted image features can be first fused to obtain a corresponding first fused feature, and then image detection is performed on the target image based on the first fused feature.

[0062] Exemplarily, the specific method used for the above-mentioned feature fusion may be feature concatenation (concat), feature addition (add), etc., which is not specifically limited in this specification.

[0063] It should be noted that this specification does not specifically limit the specific implementation of the above-mentioned image detection method based on multi-type image feature extraction. In one embodiment shown, the above-mentioned image detection method can be implemented by a trained model. Specifically, the target image to be detected can be input into a pre-trained first model, and the first model extracts preset multiple types of image features from the target image, and performs image detection on the target image based on the extracted multiple types of image features, and then outputs a first detection result.

[0064] In one illustrated embodiment, see Figure 3 , Figure 3 FIG. 1 is a schematic diagram of an image detection process based on multi-type image feature extraction provided by an exemplary embodiment. Figure 3 As shown, the first model can be a basic model structure of multi-modal multi-branch parallel input, and the first model can specifically include a central difference convolution network, a discrete cosine transform (DCT) network and an EFDMix network.

[0065] like Figure 3 As shown, the target image can be input into the center difference convolution network of the first model, and the center difference convolution network performs center difference convolution processing on the target image to obtain the differential features of the target image; in parallel, the target image (which can be a copy of the target image) can be input into the DCT transformation network of the first model, and the DCT transformation network performs discrete cosine transform processing on the target image to obtain the frequency domain features of the target image; in parallel, the target image (which can be a copy of the target image) can be input into the EFDMix network of the first model, and the EFDMix network performs EFDMix-based image enhancement processing on the target image to obtain the enhanced features of the target image.

[0066] Furthermore, if Figure 3As shown, the first model may also include a feature fusion module. Accordingly, the feature fusion module may fuse the differential features, frequency domain features and enhanced features of the target image extracted by the above-mentioned multiple parallel feature extraction branches, so as to obtain the corresponding first fusion features.

[0067] Furthermore, if Figure 3 As shown, the first model may also include a binary classification network. Accordingly, the feature fusion module may input the first fused feature obtained by fusion into the binary classification network, and the binary classification network performs image detection on the target image based on the first fused feature, for example, determining whether the user ID contained in the target image is a forged ID, or determining whether the target image is a forged facial image, etc.

[0068] It should be noted that this specification does not specifically limit the training process of the first model. In one embodiment shown, for the training of the binary classification network, considering that there may be many unknown developments in the future for AIGC forged documents, a single-center constraint (One Class Contrastive Loss Function, OCCL) can be introduced as an auxiliary loss function on the basis of the conventional binary cross entropy loss function. Among them, the basic principle of the single-center constraint is to cluster the features of known real documents as a single center, while the features of various other forged documents are learned as divergent distributions away from the single center. Based on this, when a new type of AIGC forged document appears in the future, as long as the features of the AIGC forged document are detected to be far away from the center, it will be judged as a forged document by the model and thus intercepted.

[0069] Step S203: extracting image features from the multiple image regions included in the target image respectively, and performing image detection on the target image based on the extracted image features corresponding to the multiple image regions to obtain a second detection result.

[0070] Next, image detection based on multi-region image feature extraction will be described.

[0071] In one illustrated embodiment, image features may be extracted from a plurality of image regions included in a target image, respectively, and image detection may be performed on the target image based on the extracted image features corresponding to the plurality of image regions to obtain a second detection result.

[0072] It should be noted that this specification does not specifically limit the above-mentioned multiple image areas and the image features corresponding to the multiple image areas.

[0073] In an illustrated embodiment, taking the target image as an ID card image containing a user ID card as an example, the multiple image regions may include: the full image region of the target image, the ID card region corresponding to the user ID card contained in the target image, and the background region excluding the ID card region in the full image region. Accordingly, the image features corresponding to the multiple image regions may include: full image features, ID card features, and background features; wherein the full image features (or full image representations) may include an embedding vector for representing image information contained in the full image region, the ID card features may include an embedding vector for representing image information contained in the ID card region, and the background features may include an embedding vector for representing image information contained in the background region.

[0074] In an illustrated embodiment, taking the target image as a face image containing a user's face as an example, the multiple image regions may include: a full image region of the target image, a face region corresponding to the user's face contained in the target image, and a background region excluding the face region in the full image region. Accordingly, the image features corresponding to the multiple image regions may include: full image features, face features, and background features.

[0075] In an illustrated embodiment, compared with the background area and background features, the ID area (or face area) and the ID features (or face features) may also be referred to as the foreground area and foreground features, which is not specifically limited in this specification.

[0076] In an illustrated embodiment, when performing image detection on the target image based on the extracted image features corresponding to the multiple image areas, the extracted image features corresponding to the multiple image areas can be firstly feature fused to obtain corresponding second fused features, and then image detection is performed on the target image based on the second fused features.

[0077] Exemplarily, the specific method used for the above-mentioned feature fusion may be feature concatenation (concat), feature addition (add), etc., which is not specifically limited in this specification.

[0078] It should be noted that, compared with the ID image of a real ID, the content of the background area in the ID image forged by AIGC is often limited or even single. Based on this, the present application can accurately detect whether it is a forged ID based on the background features in the ID image. In addition, the present application also integrates the full image features and ID features in the image, which can detect forged IDs more comprehensively and accurately. It should be understood that the face image forged by AIGC is the same, and will not be repeated here.

[0079] It should be noted that this specification does not specifically limit the specific implementation of the above-mentioned image detection method based on multi-region image feature extraction. In one embodiment shown, the above-mentioned image detection method can be implemented by a trained model. Specifically, the target image to be detected can be input into a pre-trained second model, and the second model extracts image features from multiple image regions contained in the target image, and performs image detection on the target image based on the extracted image features corresponding to the multiple image regions, and then outputs a second detection result.

[0080] In one illustrated embodiment, see Figure 4 , Figure 4 FIG. 1 is a schematic diagram of an image detection process based on multi-region image feature extraction provided by an exemplary embodiment. Figure 4 As shown, the second model may specifically include a ViT (Vision Transformer) model and a SAM (Segmentation-agnostic Mask) model. In one embodiment shown, still taking the target image as an ID image containing a user ID as an example, the target image may be first input into the ViT model, and the ViT model may perform feature extraction on the target image to obtain the full-image features of the target image; further, the full-image features may be input into the SAM model, and the SAM model may perform segmentation on the full-image features, thereby obtaining the ID features corresponding to the ID area, and the background features corresponding to the background area.

[0081] Next, we will combine Figure 4 The specific structures of the ViT model and the SAM model shown illustrate image detection based on multi-region image feature extraction.

[0082] like Figure 4 As shown, the ViT model can specifically include a linear transformation layer, a positional encoding, and a Transformer encoder.

[0083] In one illustrated embodiment, Figure 4 As shown in , the input target image can first be divided into multiple patches, each patch can be regarded as a "token", so as to convert the target image into sequence data for subsequent processing. Figure 4 As shown, the sizes of each block obtained by segmentation are consistent, for example, all are 16×16 pixels.

[0084] Furthermore, if Figure 4 As shown in , multiple blocks can be linearly transformed (usually a fully connected layer) to obtain an embedding vector corresponding to each block. In addition, as Figure 4As shown in the figure, in order to retain the spatial position information of each block, position encoding can also be added to the embedding vector of each block, which helps the model understand the relative position of each block.

[0085] Furthermore, if Figure 4 As shown in the figure, the embedding vectors of multiple blocks can be input into the Transformer encoder, which can include a multi-head self-attention module and a feed-forward neural network (FFN), etc., to further extract the image features of the target image and finally output the full image features (image embedding) of the target image.

[0086] like Figure 4 As shown, the SAM model can be connected with an object detector (Grounding Dino).

[0087] In one illustrated embodiment, Figure 4 As shown, first, the target image and a prompt text for indicating that the detection object is an ID area (for example, "ID card in the detection image") can be input into the object detector, and the object detector detects the ID area from the target image according to the prompt text, and outputs a bounding box corresponding to the ID area.

[0088] Furthermore, if Figure 4 As shown, the full-image features output by the Transformer encoder and the bounding box corresponding to the document area output by the Grounding Dino can be input into the SAM model. The SAM model segments the full-image features based on the bounding box corresponding to the document area, thereby obtaining the document features corresponding to the document area and the background features corresponding to the background area.

[0089] In one illustrated embodiment, Figure 4 As shown, the SAM model may specifically include a mask decoder and a prompt encoder. Specifically, Figure 4 As shown in the figure, since the Mask decoder of the SAM model can only process vectors, the bounding box corresponding to the document area output by the Grounding Dino can be first input into the Prompt encoder of the SAM model, and the Prompt encoder converts the bounding box into a corresponding vector.

[0090] Furthermore, if Figure 4As shown in the figure, the prompt encoder can input the vector of the bounding box corresponding to the document area into the mask decoder of the SAM model. In addition, the full image representation output by the Transformer encoder can also be input into the mask decoder of the SAM model. Correspondingly, the mask decoder can segment the full image features based on the above full image features and the vector of the bounding box corresponding to the document area, thereby obtaining the document features corresponding to the document area and the background features (background embedding) corresponding to the background area.

[0091] Among them, when the Mask decoder segments the document features and the background features, it can achieve segmentation by generating masks corresponding to different areas (ie, the document area and the background area), which will not be described in detail here.

[0092] Alternatively, in some possible implementations, the SAM model may not be used, but a separate segmentation module may be set up, which can directly segment the full-image features according to the bounding box corresponding to the document area output by Grounding Dino. Exemplarily, the segmentation module can first restore a feature map based on the full-image features, such as a feature map, in which each block (i.e., token) in the feature map corresponds to its position information in the map, and then use calculation methods such as Intersection over Union (IoU) to divide the tokens whose positions fall within the bounding box into the document area, and the tokens whose positions fall outside the bounding box into the background area, thereby obtaining the corresponding document features and background features, etc. This specification does not make specific limitations on this.

[0093] Furthermore, if Figure 4 As shown, the second model may also include a feature fusion module. Accordingly, the feature fusion module may fuse the extracted image features corresponding to multiple image areas (full image features, document features and background features) to obtain corresponding second fusion features.

[0094] Furthermore, if Figure 4 As shown, the second model may also include a feature comparison module. Accordingly, the feature fusion module may input the second fused feature obtained by fusion into the feature comparison module, and the feature comparison module compares the second fused feature with multiple features contained in a pre-built forged document feature library, and based on the comparison result, determines whether the user document contained in the target image is a forged document.

[0095] In an illustrated embodiment, the feature comparison module may include fully connected layers, which is not specifically limited in this specification.

[0096] In an illustrated embodiment, the forged document feature library may be a feature library constructed based on the multi-region fusion features of an existing document image containing forged documents. In an illustrated embodiment, if an image is detected as an image of a forged document by the image detection method described in step S202, or the image is manually reviewed as an image of a forged document, then based on the multi-region image feature extraction method described in step S203, image features may be extracted from multiple image regions contained in the image, and the extracted image features corresponding to the multiple image regions may be fused, and the fused features may be input into the forged document feature library, thereby constructing the forged document feature library.

[0097] In an illustrated embodiment, when determining whether the user ID contained in the target image is a forged ID based on the comparison result, it can specifically include: if the similarity between the second fusion feature and any of the multiple features contained in the forged ID feature library is greater than a preset threshold (for example, 80% or 85%, etc.), it can be determined that the user ID contained in the target image is a forged ID; otherwise, it is determined that the user ID contained in the target image is a real ID.

[0098] In an illustrated embodiment, in order to improve the efficiency of feature comparison, multiple features included in the forged document feature library may be clustered by K-means to obtain K feature clusters; wherein the K feature clusters may correspond to K central features, and the central feature corresponding to each feature cluster may be used to represent the average feature of all features included in the feature cluster; K is an integer greater than 1. Accordingly, if the similarity between the second fused feature and any central feature of the K central features is greater than a preset threshold, it may be determined that the user document included in the target image is a forged document; otherwise, it is determined that the user document included in the target image is a real document.

[0099] See also Figure 5 , Figure 5 FIG. 1 is a schematic diagram of knowledge distillation provided by an exemplary embodiment. Figure 5As shown in the figure, considering that the original ViT-based image encoder is very large in scale, it takes a long time to encode and output the full image features (image embedding). Therefore, in order to improve the timeliness of model deployment, the MobileSAM model is used to extract full image features, the coupling optimization of the image encoder and prompt encoder in the SAM model is released, and the heavyweight ViT-based (large) image encoder knowledge is distilled into a lightweight ViT-based (small) image encoder encoder. Figure 5 As shown, for the Mask decoder in the SAM model, the staff can independently choose whether to adjust it according to the actual situation and needs, and this manual does not make specific restrictions on this.

[0100] It should be noted that this specification does not specifically limit the training process of the second model. In one illustrated embodiment, the model can be supervised trained based on a large number of training samples including real documents and forged documents and their corresponding sample labels. In one illustrated embodiment, considering that there are few training samples of forged documents that can be obtained, softmax and focal loss can be used together as loss functions to train the model to solve the problem of unbalanced number of training samples of true and false categories and improve the performance of the model on minority categories.

[0101] In addition, supervised training can be performed separately for the extraction of document features and background features.

[0102] For example, taking the training of the document feature extraction part as an example, first, the training sample is input into Figure 4 In the model structure shown in FIG. 1 , the mask decoder (or a separate segmentation module) outputs the corresponding document features. Then, the background area in the training sample is masked to cover the background area (equivalent to cutting out the background area from the training sample), and the masked training sample is input into Figure 4 In the model structure shown, the full-image features output by the ViT model are the document features of the document area in the training sample. Furthermore, the full-image features output by the ViT model can be compared with the document features output by the mask decoder before, thereby achieving supervised training for the document feature extraction part.

[0103] For example, taking the training of background feature extraction as an example, first, the training sample is input into Figure 4In the model structure shown in FIG. 1 , the mask decoder (or a separate segmentation module) outputs the corresponding background features. Then, the ID area in the training sample is masked to cover the ID area (equivalent to cutting out the ID area from the training sample), and the masked training sample is input into Figure 4 In the model structure shown, the full-image features output by the ViT model are the background features of the background area in the training sample. Furthermore, the full-image features output by the ViT model can be compared with the background features output by the mask decoder before, thereby achieving supervised training for the background feature extraction part.

[0104] Step S204: In response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection, determining that the target image fails the image detection.

[0105] Further, in response to at least one of the first detection result obtained in the above step S202 and the second detection result obtained in step S203 indicating that the target image fails the image detection, it can be determined that the target image fails the image detection; otherwise, if both the first detection result and the second detection result indicate that the target image passes the image detection, it can be determined that the target image passes the image detection.

[0106] Exemplarily, taking the target image as an ID image containing a user ID, if at least one of the first detection result and the second detection result indicates that the user ID contained in the target image is a forged ID, it can be determined that the user ID contained in the target image is a forged ID.

[0107] Exemplarily, taking the target image as a facial image containing a user's face, if at least one of the first detection result and the second detection result indicates that the target image is a forged facial image, it can be determined that the target image is a forged facial image, rather than a facial image of a real user.

[0108] Corresponding to the implementation of the above method flow, the embodiment of this specification also provides an image detection device. Figure 6 , Figure 6 is a schematic diagram of the structure of an image detection device provided by an exemplary embodiment. The device 60 can be applied to Figure 1 The system architecture shown can be applied to Figure 1 Any electronic device shown. Figure 6 As shown, the device 60 includes:

[0109] An acquisition unit 601 is used to acquire a target image to be detected;

[0110] A multi-type feature extraction unit 602 is used to extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result;

[0111] A multi-region feature extraction unit 603, configured to extract image features from a plurality of image regions included in the target image, and perform image detection on the target image based on the extracted image features corresponding to the plurality of image regions, so as to obtain a second detection result;

[0112] The image detection unit 604 is configured to determine that the target image fails the image detection in response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection.

[0113] In an illustrated embodiment, the target image is a document image including a user document, and the image detection includes performing anti-counterfeiting detection on the user document included in the target image;

[0114] The image detection unit 604 is specifically used for:

[0115] In response to at least one of the first detection result and the second detection result indicating that the user ID contained in the target image is a forged ID, it is determined that the user ID contained in the target image is a forged ID.

[0116] In an illustrated embodiment, the multi-type feature extraction unit 602 is specifically used for:

[0117] The target image is input into a pre-trained first model, and the first model extracts preset multiple types of image features from the target image, and performs image detection on the target image based on the extracted multiple types of image features.

[0118] In an illustrated embodiment, the multiple types of image features include differential features of the target image; the first model includes a central differential convolutional network; the multi-type feature extraction unit 602 is specifically used to:

[0119] The target image is input into the center difference convolution network included in the first model, and the center difference convolution network performs center difference convolution processing on the target image to obtain differential features of the target image.

[0120] In an illustrated embodiment, the multiple types of image features include frequency domain features of the target image; the first model includes a discrete cosine transform network; the multi-type feature extraction unit 602 is specifically used to:

[0121] The target image is input into the discrete cosine transform network included in the first model, and the discrete cosine transform network performs discrete cosine transform processing on the target image to obtain frequency domain features of the target image.

[0122] In an illustrated embodiment, the plurality of types of image features include enhanced features based on feature distribution mixture EFDMix, the first model includes an EFDMix network; the multi-type feature extraction unit 602 is specifically used for:

[0123] The target image is input into the EFDMix network included in the first model, and the EFDMix network performs EFDMix-based image enhancement processing on the target image to obtain enhanced features of the target image.

[0124] In an illustrated embodiment, the first model includes a binary classification network; the multi-type feature extraction unit 602 is specifically used to:

[0125] Fusing the extracted image features of the plurality of types to obtain corresponding first fused features;

[0126] The first fusion feature is input into the binary classification network included in the first model, and the binary classification network determines whether the user certificate included in the target image is a forged certificate based on the first fusion feature.

[0127] In one illustrated embodiment, the loss function used by the binary classification network during training includes a binary cross entropy loss function and a single center constrained OCCL loss function; wherein the OCCL loss function is used to cluster the training samples in the training sample set of the binary classification network whose labels are real certificates in the feature space.

[0128] In an illustrated embodiment, the multi-region feature extraction unit 603 is specifically used for:

[0129] The target image is input into a pre-trained second model, and the second model extracts image features from multiple image regions contained in the target image respectively, and performs image detection on the target image based on the extracted image features corresponding to the multiple image regions.

[0130] In an illustrated embodiment, the plurality of image regions include: a full image region of the target image, an ID region corresponding to a user ID included in the target image, and a background region in the full image region excluding the ID region;

[0131] The image features corresponding to the multiple image areas include: full-image features, document features and background features; wherein the full-image features include embedded vectors for representing image information contained in the full-image area, the document features include embedded vectors for representing image information contained in the document area, and the background features include embedded vectors for representing image information contained in the background area.

[0132] In an illustrated embodiment, the second model includes a ViT model and a SAM model; the multi-region feature extraction unit 603 is specifically used to:

[0133] Inputting the target image into the ViT model, and extracting features of the target image by the ViT model to obtain full-image features of the target image;

[0134] The full image features are input into the SAM model, and the SAM model is used to segment the full image features to obtain the document features corresponding to the document area and the background features corresponding to the background area.

[0135] In an illustrated embodiment, the second model further includes an object detector; the multi-region feature extraction unit 603 is specifically configured to:

[0136] Inputting the target image and a prompt text for indicating that the detected object is the certificate area into the object detector, the object detector detecting the certificate area from the target image according to the prompt text, and outputting a bounding box corresponding to the certificate area;

[0137] The full-image features and the bounding box are input into the SAM model, and the SAM model segments the full-image features based on the bounding box.

[0138] In an illustrated embodiment, the multi-region feature extraction unit 603 is specifically used for:

[0139] Performing feature fusion on the extracted image features corresponding to the multiple image regions to obtain corresponding second fused features;

[0140] Comparing the second fused feature with a plurality of features contained in a pre-built forged document feature library, and determining whether the user document contained in the target image is a forged document based on the comparison result;

[0141] Each feature included in the forged document feature library is a fused feature obtained by fusing image features corresponding to a plurality of image regions contained in the document image and extracted from the document image containing the forged document.

[0142] In an illustrated embodiment, the multi-region feature extraction unit 603 is specifically used for:

[0143] If the similarity between the second fused feature and any one of the multiple features included in the forged document feature library is greater than a preset threshold, it is determined that the user document included in the target image is a forged document.

[0144] In an illustrated embodiment, the multi-region feature extraction unit 603 is specifically used for:

[0145] Performing K-means clustering on the multiple features contained in the forged document feature library to obtain K feature clusters; wherein the K feature clusters correspond to K central features, and the central feature corresponding to each feature cluster is used to represent the average feature of all features contained in the feature cluster; K is an integer greater than 1;

[0146] If the similarity between the second fused feature and any central feature of the K central features is greater than a preset threshold, it is determined that the user ID contained in the target image is a forged ID.

[0147] The implementation process of the functions and effects of each unit in the above-mentioned device 60 is specifically described in the above-mentioned embodiment, and will not be repeated here. It should be understood that the above-mentioned device 60 can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor (CPU) of the device where it is located reading the corresponding computer program instructions into the memory and running them. From the hardware level, in addition to the CPU and memory, the device where the above-mentioned device is located usually also includes other hardware such as chips for wireless signal transmission and reception, and / or other hardware such as boards and cards for realizing network communication functions.

[0148] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical modules, that is, they may be located in one place, or they may be distributed on multiple network modules. Some or all of the units or modules may be selected according to actual needs to achieve the purpose of the scheme of this specification. Those of ordinary skill in the art may understand and implement it without creative work.

[0149] The devices, units, and modules described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which may be in the form of a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email transceiver, a game console, a tablet computer, a wearable device, a vehicle-mounted computer, or a combination of any of these devices.

[0150] Corresponding to the above method embodiment, the embodiment of this specification also provides an electronic device. Figure 7 , Figure 7 is a schematic diagram of the structure of an electronic device provided by an exemplary embodiment. The electronic device may be Figure 1 Any electronic device in the system architecture shown. Figure 7 As shown, the electronic device includes a processor 1001 and a memory 1002, and may further include an input device 1004 (such as a keyboard, etc.) and an output device 1005 (such as a display, etc.). The processor 1001, the memory 1002, the input device 1004, and the output device 1005 may be connected via a bus or other means. Figure 7 As shown, the memory 1002 includes a computer-readable storage medium 1003, which stores a computer program that can be executed by the processor 1001. The processor 1001 can be a CPU, a microprocessor, or an integrated circuit for controlling the execution of the above method embodiment. When the processor 1001 runs the stored computer program, it can execute the various steps of the image detection method in the embodiment of this specification.

[0151] For a detailed description of each step of the above-mentioned image detection method, please refer to the previous content and will not be repeated here.

[0152] Corresponding to the above method embodiment, the embodiment of this specification also provides a computer-readable storage medium, on which a computer program is stored, and when these computer programs are executed by a processor, the various steps of the image detection method in the embodiment of this specification are executed. Please refer to the description of the above embodiment for details, which will not be repeated here.

[0153] The above description is only a preferred embodiment of this specification and is not intended to limit this specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of this specification should be included in the scope of protection of this specification.

[0154] In a typical configuration, a terminal device includes one or more CPUs, input / output interfaces, network interfaces, and memory.

[0155] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0156] Computer readable media include permanent and non-permanent, removable and non-removable media and can store information by any method or technology. The information can be computer readable instructions, data structures, program modules or other data.

[0157] Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory media such as modulated data signals and carrier waves.

[0158] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0159] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

Claims

1. An image detection method, It is characterized in that The method comprises: Acquire the target image to be detected; Extracting preset multiple types of image features from the target image respectively, and performing image detection on the target image based on the extracted multiple types of image features to obtain a first detection result; and, Extracting image features from a plurality of image regions included in the target image respectively, and performing image detection on the target image based on the extracted image features corresponding to the plurality of image regions to obtain a second detection result; In response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection, it is determined that the target image fails the image detection.

2. The method according to claim 1, It is characterized in that The target image is a certificate image containing a user certificate, and the image detection includes performing anti-counterfeiting detection on the user certificate contained in the target image; In response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection, determining that the target image fails the image detection includes: In response to at least one of the first detection result and the second detection result indicating that the user ID contained in the target image is a forged ID, it is determined that the user ID contained in the target image is a forged ID.

3. The method according to claim 2, It is characterized in that The extracting preset multiple types of image features from the target image respectively, and performing image detection on the target image based on the extracted multiple types of image features, includes: The target image is input into a pre-trained first model, and the first model extracts preset multiple types of image features from the target image, and performs image detection on the target image based on the extracted multiple types of image features.

4. The method according to claim 3, It is characterized in that The plurality of types of image features include differential features of the target image; the first model includes a central differential convolutional network; and the plurality of types of preset image features are extracted from the target image, respectively, including: The target image is input into the center difference convolution network included in the first model, and the center difference convolution network performs center difference convolution processing on the target image to obtain differential features of the target image.

5. The method according to claim 3, It is characterized in that The plurality of types of image features include frequency domain features of the target image; the first model includes a discrete cosine transform network; and the plurality of types of preset image features are extracted from the target image, respectively, including: The target image is input into the discrete cosine transform network included in the first model, and the discrete cosine transform network performs discrete cosine transform processing on the target image to obtain frequency domain features of the target image.

6. The method according to claim 3, It is characterized in that The plurality of types of image features include enhanced features based on feature distribution mixture EFDMix, and the first model includes an EFDMix network; The extracting of preset multiple types of image features from the target image respectively includes: The target image is input into the EFDMix network included in the first model, and the EFDMix network performs EFDMix-based image enhancement processing on the target image to obtain enhanced features of the target image.

7. The method according to claim 3, It is characterized in that The first model includes a binary classification network; performing image detection on the target image based on the extracted multiple types of image features includes: Fusing the extracted image features of the plurality of types to obtain corresponding first fused features; The first fusion feature is input into the binary classification network included in the first model, and the binary classification network determines whether the user certificate included in the target image is a forged certificate based on the first fusion feature.

8. The method according to claim 7, It is characterized in that The loss functions used by the binary classification network during the training process include a binary cross entropy loss function and a single center constraint OCCL loss function; wherein the OCCL loss function is used to cluster the training samples whose sample labels in the training sample set of the binary classification network are real certificates in the feature space.

9. The method according to claim 2, It is characterized in that The extracting image features from the multiple image regions included in the target image respectively, and performing image detection on the target image based on the extracted image features corresponding to the multiple image regions, comprises: The target image is input into a pre-trained second model, and the second model extracts image features from multiple image regions contained in the target image respectively, and performs image detection on the target image based on the extracted image features corresponding to the multiple image regions.

10. The method according to claim 9, It is characterized in that The multiple image regions include: a full image region of the target image, an ID region corresponding to a user ID included in the target image, and a background region in the full image region excluding the ID region; The image features corresponding to the multiple image areas include: full-image features, document features and background features; wherein the full-image features include embedded vectors for representing image information contained in the full-image area, the document features include embedded vectors for representing image information contained in the document area, and the background features include embedded vectors for representing image information contained in the background area.

11. The method according to claim 10, It is characterized in that The second model includes a ViT model and a SAM model; and extracting image features from a plurality of image regions included in the target image respectively includes: Inputting the target image into the ViT model, and extracting features of the target image by the ViT model to obtain full-image features of the target image; The full image features are input into the SAM model, and the SAM model is used to segment the full image features to obtain the document features corresponding to the document area and the background features corresponding to the background area.

12. The method according to claim 11, It is characterized in that The second model further includes an object detector; the step of inputting the full image features into the SAM model, and segmenting the full image features by the SAM model, comprises: Inputting the target image and a prompt text for indicating that the detected object is the certificate area into the object detector, the object detector detecting the certificate area from the target image according to the prompt text, and outputting a bounding box corresponding to the certificate area; The full-image features and the bounding box are input into the SAM model, and the SAM model segments the full-image features based on the bounding box.

13. The method according to claim 9, It is characterized in that The performing image detection on the target image based on the extracted image features corresponding to the multiple image regions comprises: Performing feature fusion on the extracted image features corresponding to the multiple image regions to obtain corresponding second fused features; Comparing the second fused feature with a plurality of features contained in a pre-built forged document feature library, and determining whether the user document contained in the target image is a forged document based on the comparison result; Each feature included in the forged document feature library is a fused feature obtained by fusing image features corresponding to a plurality of image regions contained in the document image and extracted from the document image containing the forged document.

14. The method according to claim 13, It is characterized in that The determining, based on the comparison result, whether the user certificate included in the target image is a forged certificate includes: If the similarity between the second fused feature and any one of the multiple features included in the forged document feature library is greater than a preset threshold, it is determined that the user document included in the target image is a forged document.

15. The method according to claim 14, It is characterized in that If the similarity between the second fused feature and any one of the multiple features included in the forged document feature library is greater than a preset threshold, determining that the user document included in the target image is a forged document includes: Performing K-means clustering on the multiple features contained in the forged document feature library to obtain K feature clusters; wherein the K feature clusters correspond to K central features, and the central feature corresponding to each feature cluster is used to represent the average feature of all features contained in the feature cluster; K is an integer greater than 1; If the similarity between the second fused feature and any central feature of the K central features is greater than a preset threshold, it is determined that the user ID contained in the target image is a forged ID.

16. An image detection device, It is characterized in that The device comprises: An acquisition unit, used for acquiring a target image to be detected; a multi-type feature extraction unit, configured to extract preset multiple types of image features from the target image, and perform image detection on the target image based on the extracted multiple types of image features to obtain a first detection result; a multi-region feature extraction unit, configured to extract image features from a plurality of image regions included in the target image, respectively, and perform image detection on the target image based on the extracted image features corresponding to the plurality of image regions, so as to obtain a second detection result; An image detection unit is used to determine that the target image fails the image detection in response to at least one of the first detection result and the second detection result indicating that the target image fails the image detection.

17. An electronic device, It is characterized in that include: Memory and processor; The memory has stored thereon computer programs / instructions executable by the processor; When the processor runs the computer program / instructions, the method according to any one of claims 1 to 15 is performed.

18. A computer-readable storage medium, It is characterized in that A computer program / instruction is stored thereon, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

19. A computer program product, It is characterized in that The computer program product comprises a computer program / instruction, and when the computer program / instruction is executed by a processor, the method according to any one of claims 1 to 15 is implemented.

Images