Method, device, computer-readable storage medium, electronic device, and computer program product for generating a credential image sample

Abstract

The embodiments of this specification disclose a credential image sample generation method, device, computer-readable storage medium, electronic device and computer program product. The credential image sample generation method may include: after obtaining the initial target object image contained in the user credential image, a generative artificial intelligence model can be used to perform image drawing processing on the area where the target object is located or the background area in the initial target object image to obtain a high-quality fictitious target object image, thereby replacing the initial target object image in the user credential image with the fictitious target object image, thereby efficiently and conveniently generating a large number of credential image samples with good authenticity and offensive nature.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for generating a credential image sample, a computer-readable storage medium, an electronic device, and a computer program product. Background Art

[0002] Risk control can help regulators and enterprises discover and resolve risks faced in the business operation process in advance, which is conducive to protecting the legitimate rights and interests of all parties, so that all walks of life have begun to pay more and more attention to risk control to cope with various internal and external risk challenges. At present, users may use image processing technology to tamper with or synthesize offensive user credential images, so as to use offensive user credential images to illegally obtain benefits from service providers. Based on this, service providers often need to use a large number of offensive credential image samples and trusted credential image samples to train deep learning models during business operations, so as to use the trained deep learning models to distinguish whether the credential images submitted by users are authentic. However, since the user credential images obtained during business operations are usually trusted credential image samples, it is difficult to obtain a sufficient number of offensive credential image samples, which cannot meet the model training requirements. Summary of the invention

[0003] The embodiments of the present specification provide a credential image sample generation method, device, computer-readable storage medium, electronic device and computer program product, which can efficiently and conveniently generate a large number of credential image samples with good authenticity and offensive nature, so as to reduce the difficulty of obtaining high-quality credential image samples with offensive nature, and at the same time, it is conducive to improving the classification accuracy of whether the user credential image is credible by the credential image classification model trained by using the above-mentioned credential image samples with offensive nature.

[0004] An embodiment of the present specification provides a method for generating a credential image sample, the method comprising:

[0005] Get the user credential image;

[0006] determining an initial target object image contained in the user credential image;

[0007] Using a generative artificial intelligence model, image rendering is performed on a target object region or a background region in the initial target object image to obtain a fictitious target object image;

[0008] The initial target object image in the user credential image is replaced by the fictitious target object image to obtain a credential image sample.

[0009] The embodiment of this specification also provides a voucher image classification model training method, including:

[0010] Acquire a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the credential image sample generation method described above;

[0011] The training sample set is used to perform model training on the initial voucher image classification model to obtain the voucher image classification model.

[0012] The embodiment of this specification also provides a voucher image classification method, including:

[0013] Get the user's target credential image;

[0014] The target credential image is classified by using a credential image classification model to obtain a classification result reflecting whether the target credential image is a credential image; wherein the credential image classification model is a model trained by using the above-mentioned credential image classification model training method.

[0015] The embodiment of the present specification also provides a credential image sample generating device, including:

[0016] An acquisition module, used to acquire a user credential image;

[0017] a determination module, configured to determine an initial target object image contained in the user credential image;

[0018] An image processing module, used to use a generative artificial intelligence model to perform image rendering processing on a target object area or a background area in the initial target object image to obtain a fictitious target object image;

[0019] The sample generation module is used to replace the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample.

[0020] The embodiment of this specification also provides a voucher image classification model training device, including:

[0021] An acquisition module, configured to acquire a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the credential image sample generation method described above;

[0022] The training module is used to perform model training on the initial voucher image classification model using the training sample set to obtain the voucher image classification model.

[0023] The embodiment of this specification also provides a voucher image classification device, including:

[0024] An acquisition module, used to acquire a target credential image of a user;

[0025] A classification module is used to classify the target credential image using a credential image classification model to obtain a classification result that reflects whether the target credential image is a trusted credential image; wherein the credential image classification model is a model trained using the above-mentioned credential image classification model training method.

[0026] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the above method are implemented.

[0027] An embodiment of the present specification also provides an electronic device, comprising: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the above method.

[0028] The embodiments of the present specification also provide a computer program product having at least one instruction stored thereon, and the at least one instruction implements the steps of the above method when executed by a processor.

[0029] In an embodiment of the present specification, an initial target object image contained in a user credential image can be obtained, and a generative artificial intelligence model can be used to perform image drawing processing on the target object area or background area in the initial target object image to obtain a high-quality fictitious target object image, thereby replacing the initial target object image in the user credential image with the fictitious target object image, thereby efficiently and conveniently generating a large number of credential image samples with offensive properties with good authenticity, which is conducive to reducing the difficulty of obtaining high-quality credential image samples with offensive properties. At the same time, it can better meet the training requirements of the deep learning model required for risk detection of user credential images, thereby also helping to improve the classification accuracy of the credential image classification model trained using the above-mentioned credential image samples with offensive properties for whether the user credential image is credible, so as to ensure the risk control effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of an application scenario of a credential image sample generation solution provided in an embodiment of this specification;

[0031] Figure 2 A flowchart of a method for generating a credential image sample provided in an embodiment of this specification;

[0032] Figure 3A schematic diagram of a user credential image provided in an embodiment of this specification;

[0033] Figure 4 A schematic diagram of the generation principle of a user credential image provided in an embodiment of this specification;

[0034] Figure 5 A schematic diagram of another principle of generating a user credential image provided in an embodiment of this specification;

[0035] Figure 6 A flowchart of a method for training a voucher image classification model provided in an embodiment of this specification;

[0036] Figure 7 A flowchart of a voucher image classification method provided in an embodiment of this specification;

[0037] Figure 8 A schematic diagram of the structure of a credential image sample generating device provided in an embodiment of this specification;

[0038] Fig. 9 A schematic diagram of the structure of a voucher image classification model training device provided in an embodiment of this specification;

[0039] Fig.10 A schematic diagram of the structure of a voucher image classification device provided in an embodiment of this specification;

[0040] Fig.11 A schematic diagram of the structure of an electronic device provided in an embodiment of this specification. DETAILED DESCRIPTION

[0041] In order to make the purpose, technical solutions and advantages of this specification more clear, the technical solutions of this specification will be clearly and completely described below in combination with the specific embodiments of this specification and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of this specification, not all of them. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this specification.

[0042] Before specifically describing the credential image sample generation solution provided in the embodiments of this specification, the relevant technical background is first described.

[0043] At present, in order to prevent the risk of users using untrusted credential images to obtain services, service providers usually need to use a large number of attacking credential image samples and trusted credential image samples to train deep learning models, so as to use the trained deep learning models to distinguish whether the credential images submitted by users are authentic. Since user credential images obtained during business operations are usually trusted credential image samples, it is usually easier to obtain trusted credential image samples.

[0044] However, due to the small number of offensive credential image samples obtained during business operations, the model training process usually cannot meet the demand for offensive credential image samples. Based on this, some service providers will pre-make credential templates and paste the acquired target object images into the target object area in the credential template as offensive credential image samples. The offensive credential image samples generated in the above manner have obvious signs of forgery, and the difference between them and the offensive credential images actually used by users is also large, which makes the accuracy of the credential image authenticity recognition results generated by the deep learning model trained using these credential image samples poor and cannot meet the risk control needs.

[0045] Based on this, how to efficiently and conveniently generate a large number of credential image samples with good authenticity and offensive nature, reduce the difficulty of obtaining high-quality credential image samples with offensive nature, and ensure the accuracy of the trained credential image classification model in classifying whether the user credential image is credible, has become an urgent problem to be solved.

[0046] See also Figure 1 , is a schematic diagram of an application scenario of a credential image sample generation solution provided in an embodiment of this specification.

[0047] like Figure 1 As shown, the application installed at the device 11 can be used to extract the initial target object image (for example, a car image) contained in the user credential image 12; then, the generative artificial intelligence model is used to perform image rendering processing on the target object area (for example, the area where the car is located) or the background area (for example, the area around the car) in the initial target object image to obtain a high-quality fictitious target object image; after replacing the initial target object image in the user credential image with the fictitious target object image, a large number of credential image samples 13 with good authenticity and offensive nature can be obtained.

[0048] Subsequently, the above-mentioned attacking credential image sample 13 can be used as a negative sample, and the trusted user credential image can be used as a positive sample, so as to use the training sample set containing the above-mentioned positive and negative samples to train the initial credential image classification model, thereby facilitating the improvement of the classification accuracy of the trained credential image classification model for whether the user credential image is credible, so as to ensure the risk control effect.

[0049] See also Figure 2 , is a flowchart of a method for generating a credential image sample provided in an embodiment of the present specification. The execution subject of the process may be an application for generating a credential image sample, or a device equipped with an application for generating a credential image sample. Figure 2 The process shown in the figure is described in detail, and the method for generating a credential image sample may specifically include the following steps:

[0050] Step 202, obtaining a user credential image.

[0051] In the embodiment of the present specification, when the service provider has the need to verify the specified type of user credentials, it is usually necessary to obtain a large number of authentic and reliable user credential image samples and user credential image samples with offensive properties to train a deep learning model for verifying the authenticity of the specified type of user credentials. Based on this, the image of the specified type of user credentials can be obtained as the user credential image mentioned in step 202, so as to generate the credential image sample with offensive properties corresponding to the specified type of user credentials by processing the user credential image.

[0052] Among them, the type of credential to which the user credential image belongs can be set according to actual needs. For example, it can include but is not limited to: a credential used to reflect the ownership of a motor vehicle (such as a motor vehicle driving license), a credential used to reflect the entry and exit rights of a vehicle or a user, a credential used to reflect the user's identity, a credential used to reflect the user's academic qualifications, a credential used to reflect the user's professional qualifications, a credential used to reflect the user's assets, etc., without specific limitation.

[0053] In practical applications, the user credential image obtained in step 202 can be either a genuine and authentic user credential image submitted by the user to the service provider in the process of obtaining services, or a user credential image with offensive nature submitted by the user. Of course, it can also be a genuine and authentic or offensive user credential image obtained by the service provider in other ways, which is not specifically limited.

[0054] Step 204: Determine the initial target object image included in the user credential image.

[0055] In the embodiments of the present specification, the user credential image may generally include an initial target object image, and the user generally needs to tamper with the initial target object image included in the user credential image in order to use the tampered credential image to illegally obtain services from the service provider. Therefore, when generating a user credential image sample with an attacking nature, the target detection technology may be first used to determine the initial target object image included in the user credential image, so as to modify the initial target object image included in the user credential image, thereby obtaining a credential image sample with an attacking nature.

[0056] Step 206, using a generative artificial intelligence model, performing image rendering processing on the target object area or the background area in the initial target object image to obtain a fictitious target object image.

[0057] In the embodiments of this specification, artificial intelligence generated content (AIGC) may refer to the technology of using artificial intelligence algorithms and models to generate pictures, texts, sounds, videos, codes and other contents with appropriate generalization capabilities through learning and identifying existing data. Unlike traditional artificial intelligence, the generative artificial intelligence model can not only process the input data, but also learn and simulate the internal laws of things, so that it can autonomously create new realistic content with certain logic and coherence based on the input conditions or guidance information.

[0058] Based on this, a generative artificial intelligence model can be used to perform image drawing processing on the area where the target object is located or the background area in the initial target object image, thereby retaining some of the original image content in the initial target object image and automatically generating some new high-quality image content using the generative artificial intelligence model, which is conducive to ensuring the quality of the generated fictional target object image with an offensive nature.

[0059] In practical applications, the target objects contained in the target object region in the initial target object image may be of various types, for example, the target objects may include at least one of: a credential anti-counterfeiting object, a user biometric object, and a physical object certified by a credential. The credential anti-counterfeiting object may include an anti-counterfeiting pattern, a seal of a designated institution, a logo, etc.; the user biometric object may include a user's fingerprint, iris, face, etc.; the physical object certified by a credential may include: a physical object such as a house or a vehicle; and there is no specific limitation on this.

[0060] Step 208: Use the fictitious target object image to replace the initial target object image in the user credential image to obtain a credential image sample.

[0061] In the embodiment of the present specification, the original target object image in the user credential image is replaced with the fictitious target object image with good authenticity generated in step 206, so as to obtain a high-quality credential image sample with attacking properties, which is convenient and quick.

[0062] Figure 2 The method can obtain the initial target object image contained in the user credential image from the user credential image, and use the generative artificial intelligence model to perform image drawing processing on the target object area or background area in the initial target object image to obtain a high-quality fictitious target object image, so as to replace the initial target object image in the user credential image with the fictitious target object image, so as to efficiently and conveniently generate a large number of credential image samples with good authenticity and offensive nature, which is conducive to reducing the difficulty of obtaining high-quality credential image samples with offensive nature, and thus can better meet the training requirements of the deep learning model required for risk detection of user credential images, and is also conducive to improving the classification accuracy of whether the user credential image is credible by the credential image classification model trained by using the above-mentioned credential image samples with offensive nature, so as to ensure the risk control effect.

[0063] based on Figure 2 The method in this specification also provides some specific implementation plans of the method, which are described below.

[0064] In one feasible implementation, the generative artificial intelligence model may include an image restoration model.

[0065] Correspondingly, step 206: using a generative artificial intelligence model to perform image rendering processing on the background area in the initial target object image to obtain a fictitious target object image may include:

[0066] An image segmentation model is used to perform image segmentation processing on the initial target object image to obtain a mask image used to mask the area where the target object is located in the initial target object image.

[0067] The image restoration model is used to perform background region restoration processing based on the initial target object image and the mask image to obtain a background restoration image.

[0068] A fictitious target object image is generated according to the background restoration image and an image in a region where the target object is located in the initial target object image.

[0069] In the embodiments of the present specification, the initial target object image in the user credential image can be divided into: an area containing the target object and a background area. Since the information in the background area can not only provide context clues to assist in understanding the authenticity and consistency of the foreground target object, but also may carry traces of editing and forgery clues, the background area often contains key feature data for authenticity identification of the credential image, so that the background image in the initial target object image can be combined to synthesize a fictitious target object image to improve the authenticity and quality of the fictitious target object image. However, due to the limited area of ​​the background area in the initial target object image, it is often impossible to combine the background image in the initial target object image well to generate a fictitious target object image, which affects the authenticity and quality of the generated fictitious target object image.

[0070] Image inpainting can refer to the process of reconstructing missing or damaged areas in an image. By replacing or filling the texture / pixels of a specified area in an image so that it blends in with the surrounding texture / pixels, a natural and harmonious restoration effect can be achieved. Currently, image inpainting technology has been applied to a variety of scenarios, such as repairing and filling cracks, scratches, and blemishes in old photos to obtain complete and authentic images. Also, removing unnecessary image content and filling the gaps with reasonable image content to obtain an image that meets the needs and has good authenticity.

[0071] Based on this, the image segmentation model can be used to perform image segmentation processing on the initial target object image to obtain a mask image used to mask the area where the target object is located in the initial target object image. After the initial target object image and the mask image are input into the image restoration model built based on deep learning technology, the image restoration model can remove the target object image in the initial target object image and then perform image restoration processing on the remaining background area. At this time, the image restoration model can output a background restoration image with a larger area and better authenticity.

[0072] For ease of understanding, Figure 3 A schematic diagram of a user credential image provided in an embodiment of this specification. Figure 3 As shown, the user credential image 301 may include an initial target object image 302, and the initial target object image 302 may include a target object region 303 (e.g., a region where a car is located) and a background region 304 (e.g., a region surrounding the car). After image segmentation processing is performed on the initial target object image 302, a mask image ( Figure 3 (not shown), the mask image and the initial target object image 302 are input into the image restoration model, and then the background restoration image output by the image restoration model can be obtained.

[0073] In the embodiments of the present specification, after obtaining the background restoration image, an image in any area can be intercepted from the background restoration image to obtain a new background image with better authenticity, so as to use this new background image and the image in the area where the target object is located in the initial target object image to synthesize a fictitious target object image. At this time, since the quality of the background image and the target object image used to synthesize the fictitious target object image is good, it is beneficial to ensure the quality of the fictitious target object image. In addition, since a large number of background images with differentiation and good authenticity can be generated using a single background restoration image, the demand for generating a large number of high-quality fictitious target object images can also be met.

[0074] In practical applications, there may be many types of image restoration models, for example, including but not limited to: LaMa (large mask inpainting) model, BrushNet model, MAT (Mask-Aware Transformer for Large Hole Image Inpainting) model, etc., without specific limitation.

[0075] In a feasible implementation manner, the user credential image acquired in step 202 may include: a trusted user credential image and an untrusted user credential image.

[0076] Correspondingly, generating a fictitious target object image according to the background restoration image and the image in the region where the target object is located in the initial target object image may include:

[0077] Generate a fictitious target object image based on the background repair image generated based on any of the untrusted user credential images and an image in the area where the target object is located in any of the trusted user credential images; and / or,

[0078] A fictitious target object image is generated according to the background repair image generated based on any one of the trusted user credential images and an image in the area where the target object is located in any one of the untrusted user credential images.

[0079] In the embodiments of this specification, the trusted user credential image may refer to a user credential image obtained by performing image acquisition on the user's trusted credential, and the service provider will usually determine that these trusted user credential images can pass the credential image authenticity detection. The untrusted user credential image may refer to an offensive user credential image generated by image tampering technology or other attack methods, and the service provider will usually determine that these untrusted user credential images are prohibited from passing the credential image authenticity detection.

[0080] Since the background area of ​​the untrusted user credential image often contains key offensive feature data used for credential image authenticity identification, and the authenticity of the target object image in the area where the target object in the trusted user credential image is located is relatively good, the background repair image generated based on the untrusted user credential image can be synthesized with the target object image in the trusted user credential image to obtain a high-quality fictional target object image.

[0081] Of course, the background repair image generated based on the trusted user credential image and the target object image in the untrusted user credential image may also be synthesized to obtain a fictitious target object image, which is beneficial to improving the richness of the types and quantity of the generated fictitious target object images.

[0082] In practical applications, the location of the target object region in the initial target object image often needs to meet preset requirements, for example, the target object region in the initial target object image should be in the middle of the image, or the distance between the target object region and the top or side of the image should be in a preset interval, etc. Therefore, in order to improve the authenticity of the synthesized fictitious target object image, the location of the target object region in the fictitious target object image can also meet the above preset requirements, which will not be elaborated.

[0083] In a feasible implementation manner, generating a fictitious target object image according to the background restoration image and the image in the region where the target object is located in the initial target object image may include:

[0084] Performing image stitching processing on the background restoration image and the image in the area where the target object is located in the initial target object image to obtain a fictitious target object image; or,

[0085] An image fusion process is performed on the background restoration image and the image in the region where the target object is located in the initial target object image to obtain a fictitious target object image.

[0086] In the embodiments of the present specification, an image stitching technology or an image fusion technology may be used to synthesize a fictitious target object image based on a background restoration image and an image in a region where the target object is located in an initial target object image.

[0087] Among them, the image stitching technology can directly paste the image in the target object area in the initial target object image onto the background repair image to obtain a fictitious target object image; or, it can first determine the display area of ​​the image in the target object area in the initial target object image in the background repair image, and then use the pixel value of each pixel in the image in the target object area to replace the original pixel value at the corresponding position in the display area in the background repair image to obtain a fictitious target object image, which is convenient and quick; there is no specific limitation on this.

[0088] Image fusion technology can refer to the technology of integrating multiple source images (which may come from different sensors, different perspectives, and different time points) into a single output image after image processing, thereby effectively merging the complementary, redundant or unique information in each source image to generate a high-quality integrated image that contains all key details and has enhanced characteristics. Since image fusion technology can eliminate the problems of unnatural boundaries between foreground and background, mismatched color and lighting information between foreground and background, missing or unreasonable foreground shadows or reflections, etc. existing in image stitching technology, the authenticity and quality of the obtained fictitious target object image can be improved based on image fusion technology.

[0089] In practical applications, the Poisson fusion algorithm can be used to perform image fusion processing on the background repair image and the image in the target object area of ​​the initial target object image to obtain a fictitious target object image. The pre-trained deep learning model can also be used to perform image fusion processing on the background repair image and the image in the target object area to obtain a fictitious target object image, and there is no specific limitation on this.

[0090] In one feasible implementation, the generative artificial intelligence model includes a diffusion model.

[0091] Correspondingly, step 206: using a generative artificial intelligence model to perform image rendering processing on the region where the target object is located in the initial target object image to obtain a fictitious target object image may include:

[0092] An image segmentation model is used to perform image segmentation processing on the initial target object image to obtain a mask image used to mask the area where the target object is located in the initial target object image.

[0093] The diffusion model is used to process the initial target object image, the mask image and the preset prompt words to obtain a fictitious target object image output by the diffusion model; wherein the preset prompt words are used to guide the diffusion model to generate a target object image that conforms to a preset style in the area where the target object is located.

[0094] In the embodiments of this specification, diffusion models are a type of generative model that can creatively generate high-fidelity output data by gradually injecting noise into data samples and then training the model to learn to remove the noise. Due to its powerful image generation capabilities, diffusion models have been widely used in content creation, smart marketing, visual design and other fields.

[0095] Prompts can help indicate the direction and theme of the current data processing task, ensuring that the AI ​​model can understand the specific needs of the user, so as to improve the relevance and accuracy of the AI ​​model's response. At present, some diffusion models can receive prompts to describe the style of the output image, so as to generate images with a certain style through the context learning ability of the diffusion model.

[0096] Based on this, preset prompt words used to guide the diffusion model to generate a target object image that meets the preset style can be input into the diffusion model. The initial target object image and the mask image used to mask the target object area in the initial target object image are input into the diffusion model. The diffusion model can retain the background area in the initial target object image that is not masked by the mask image, and perform feature extraction and image generation processing on the target object in the area where the preset prompt words and the target object masked by the mask image are located. The diffusion model will also fuse the target object image generated by it that meets the preset style with the image in the background area retained in the early stage, so as to finally output a fictional target object image that meets the preset style.

[0097] In practical applications, the specific content and format of the preset prompt words can be set according to actual needs, and the preset prompt words can be text data or image data. For example, assuming that the target object in the target object area is a car, the preset prompt words can be used to indicate the generation of a car of a specified color, a specified model or a specified brand, or the preset prompt words can also be a car image that meets the preset style. When the target object in the target object area is a human face, the preset prompt words can be used to indicate the generation of a young man's face image, or to indicate the generation of a face image with long hair, a high nose bridge, and big eyes, or to indicate the generation of a face image of a person wearing a black top and not wearing a hat, etc., without specific limitations. By inputting the preset prompt words into the diffusion model, it is beneficial to improve the variety of fictional target object images output by the diffusion model, reduce the dependence on the original target object image, and thus better meet the needs of efficiently generating a large number of high-quality fictional target object images.

[0098] In practical applications, there may be many types of diffusion models, for example, including but not limited to: BrushNet model, MimicBrush model, Stable Diffusion model, etc., without specific limitation. In addition, since the diffusion model can also be used to build a large model for performing image generation tasks, the diffusion model can also include a multimodal large model or a single-modal large model with image generation capabilities. And, in some cases, only the initial target object image and at least one of the preset prompt words can be input into the diffusion model without inputting the mask image into the diffusion model, and the fictitious target object image output by the diffusion model can also be obtained, without specific limitation.

[0099] In a feasible implementation, step 208: replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample may include:

[0100] Determine location information of the initial target object image in the user credential image.

[0101] Based on the position information of the initial target object image, image stitching processing is performed on the fictitious target object image and the user credential image to obtain a first credential image sample; wherein the position of the fictitious target object image in the first credential image sample is consistent with the position of the initial target object image in the user credential image; or,

[0102] Based on the position information of the initial target object image, image fusion processing is performed on the fictitious target object image and the user credential image to obtain a second credential image sample; wherein the position of the fictitious target object image in the second credential image sample is consistent with the position of the initial target object image in the user credential image.

[0103] In the embodiments of this specification, image stitching technology or image fusion technology can be used to replace the initial target object image in the user's credential image with a fictitious target object image to obtain a credential image sample. Since the implementation principles of image stitching technology and image fusion technology have been introduced above, they will not be repeated here.

[0104] In practical applications, the position of the initial target object image in the user credential image may also need to meet preset conditions, for example, the initial target object image may need to be located in the left area of ​​the user credential image, or in the top center area, etc. Based on this, in order to ensure the authenticity of the generated offensive credential image sample, when synthesizing the credential image sample using the fictitious target object image and the user credential image, it is necessary to first determine the position information of the initial target object image in the user credential image, and then, combined with the position information of the initial target object image, display the fictitious target object image in the area where the initial target object image is located, so that the position of the fictitious target object image contained in the synthesized first / second credential image sample is consistent with the position of the initial target object image in the user credential image, which will not be elaborated on.

[0105] When an image acquisition device is used to acquire an image of a user credential, the acquired credential image is often distorted, thereby affecting the accuracy of the credential information it carries. Based on this, image correction technology can be used to generate a user credential image with less distortion, so as to improve the quality of the credential image sample generated based on the user credential image. Also, since the initial user credential image acquired by the image acquisition device contains not only the user credential image but also the environmental information of the user credential, the first / second credential image sample with offensive properties generated by the above embodiment can also be combined with the environmental information of the user credential, thereby improving the richness of the user credential sample with offensive properties that can be finally obtained.

[0106] Based on this, in a feasible implementation, step 202: obtaining a user credential image may include:

[0107] Get the initial user credential image.

[0108] Perform image correction processing on the initial user credential image to obtain a corrected image.

[0109] A credential corner point detection process is performed on the corrected image to obtain the user credential image contained in the corrected image.

[0110] The step of replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample may further include:

[0111] The user credential image contained in the corrected image is replaced with the first credential image sample or the second credential image sample to obtain a third credential image sample. And / or,

[0112] According to the affine transformation relationship between the initial user credential image and the corrected image, an affine transformation process is performed on the third credential image sample to obtain a fourth credential image sample having a posture consistent with that of the initial user credential image.

[0113] In the embodiments of this specification, the initial user credential image may refer to a trusted image obtained by collecting the user credential, or a forged image including the user credential generated by using image processing technology, and no specific limitation is made to this.

[0114] Since the frontal perspective refers to the image that can be captured for the user credential when the user credential and the image acquisition device are roughly parallel. Therefore, the credential image under the frontal perspective is usually closest to the original appearance of the user credential in terms of visual perception, and the distortion is also the slightest. Based on this, the initial user credential image can be subjected to image correction processing to obtain the credential image under the frontal perspective, thereby serving as the corrected image. Subsequently, after detecting the positions of the corner points of each document contained in the corrected image, the area where the user credential is located can be determined in combination with the positions of the corner points of each document, and then the user credential image mentioned in step 202 can be extracted from the corrected image.

[0115] For ease of understanding, Figure 4 The following is a schematic diagram of the principle of generating a user credential image provided in the embodiment of this specification. Figure 4 As shown, after image correction processing is performed on the initial user credential image (a), a corrected image (b) can be obtained, and credential corner point detection processing is performed on the corrected image (b) to obtain the user credential image (c) contained in the corrected image (b).

[0116] In the embodiment of the present specification, in addition to the user credential image, the corrected image may also include a background area for reflecting the environment in which the user credential is located (for example, the area containing a large tree in the corrected image (b)). Since these background areas often carry the feature data required for identifying the authenticity of the credential image, the first / second credential image samples generated above can also be used to replace the user credential image originally contained in the corrected image to obtain a third credential image sample with an aggressiveness under a wide viewing angle. In addition, according to the affine transformation relationship between the initial user credential image (a) and the corrected image (b), an affine transformation process can be performed on the third credential image sample to obtain a fourth credential image sample with an aggressiveness under a wide viewing angle that is consistent with the posture of the initial user credential image (a). Subsequently, the first credential image sample, the second credential image sample, the third credential image sample, and the fourth credential image sample can be used to train a credential image classification model for distinguishing the authenticity of the target credential image submitted by the user. The solution in this embodiment is conducive to further improving the generation capability and type richness of credential image samples with good authenticity and offensive nature, and is convenient, fast and practical, thereby also helping to improve the classification accuracy of the trained credential image classification model.

[0117] In a feasible implementation manner, performing image correction processing on the initial user credential image to obtain a corrected image may include:

[0118] The pose information of the initial user credential image is determined by using a pose estimation model based on scale-invariant feature transformation.

[0119] A first transformation matrix between the initial user credential image and the frontal perspective credential image is determined according to the pose information of the initial user credential image.

[0120] Based on the first transformation matrix, an affine transformation process is performed on the initial user credential image to obtain the corrected image.

[0121] In the embodiments of this specification, the scale-invariant feature transform (SIFT) is a feature point detection algorithm used in image processing. Since SIFT feature points are invariant to rotation, scale scaling, and brightness changes, these characteristics enable the SIFT feature points in the image to have a certain stability when the image perspective changes and affine transformation is processed, so that a pose estimation model based on the scale-invariant feature transform can be pre-built and trained to accurately evaluate the pose information of the initial user credential image using the pose estimation model based on the scale-invariant feature transform.

[0122] In practical applications, the training samples for the pose estimation model based on scale-invariant feature transformation can be credential image samples in various poses, and the pose information of each credential image sample can also be used as the corresponding label data. By training the pose estimation model with credential image samples carrying label data, the pose estimation model can be enabled to accurately output the pose information of the initial user credential image. This will not be elaborated here.

[0123] Since the image correction processing is performed on the initial user credential image, it is essentially to perform affine transformation processing on the initial user credential image to obtain a credential image under the frontal perspective as the corrected image. Therefore, the first transformation matrix between the initial user credential image and the frontal perspective credential image can be determined based on the posture information of the initial user credential image. So as to perform affine transformation processing on the initial user credential image according to the first transformation matrix to obtain the corrected image. In practical applications, the first transformation matrix between the initial user credential image and the frontal perspective credential image is also equivalent to the affine transformation relationship between the initial user credential image and the corrected image.

[0124] Therefore, the first transformation matrix may be used to generate a fourth voucher image sample later, which will not be described in detail.

[0125] In the embodiment of this specification, by using the pose estimation model based on scale-invariant feature transformation, the pose information of the initial user credential image with good accuracy can be obtained, and then the image affine transformation processing can be performed in combination with the pose information to obtain a corrected image under the frontal perspective, so that a high-quality user credential image can be extracted from the corrected image. It can be seen that the scheme in this embodiment can obtain a high-quality user credential image from the initial user credential image collected at any angle, so there is no need to limit the shooting perspective of the initial user credential image, which is conducive to reducing the difficulty of collecting the initial user credential image, and then reducing the difficulty of obtaining the user credential image mentioned in step 202, and has good practicality.

[0126] In a feasible implementation, step 202: obtaining a user credential image may include:

[0127] Get the initial user credential image.

[0128] A credential corner point detection process is performed on the initial user credential image to obtain an initial credential image.

[0129] Performing image correction processing on the initial credential image to obtain the user credential image.

[0130] The step of replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample may further include:

[0131] According to the affine transformation relationship between the initial user credential image and the user credential image, an affine transformation process is performed on the first credential image sample or the second credential image sample to obtain a fifth credential image sample having the same posture as the initial credential image. And / or,

[0132] The initial credential image included in the initial user credential image is replaced with the fifth credential image sample to obtain a sixth credential image sample.

[0133] In the embodiments of this specification, based on the credential corner detection technology, the area where the user credential is located can be first determined from the distorted initial user credential image, and the image in this area can be extracted to obtain the distorted initial credential image. Then, the distorted initial credential image can be corrected to obtain an image from the frontal perspective, so as to serve as a high-quality user credential image with good flexibility.

[0134] For ease of understanding, Figure 5 This is another schematic diagram of the principle of generating a user credential image provided in the embodiment of this specification. Figure 5 As shown, credential corner detection can be performed on the initial user credential image (a) to obtain a distorted initial credential image (b), and image correction processing can be performed on the initial credential image (b) to obtain a user credential image (c) under a frontal perspective.

[0135] In the embodiment of the present specification, the initial user credential image (a) may include, in addition to the initial credential image (b), a background area for reflecting the environment in which the user credential is located (for example, an area containing a large tree in the initial user credential image (a)). Since these background areas often carry feature data required for identifying the authenticity of the credential image, the first / second credential image samples generated above may also be combined with the initial user credential image (a) to obtain an aggressive credential image sample with a large field of view.

[0136] Specifically, since the initial user credential image (a) is distorted, and the user credential image (c), the first / second credential image samples, etc. are all images under the frontal perspective, it is necessary to perform affine transformation processing on the first / second credential image samples according to the affine transformation relationship between the initial user credential image (a) and the user credential image (c), so as to obtain a fifth credential image sample that is consistent with the posture of the initial credential image (b) in the initial user credential image (a). Then, based on the image splicing processing method or the image fusion processing method, the fifth credential image sample is used to replace the initial credential image (b) in the initial user credential image (a) to obtain an aggressive sixth credential image sample under a wide perspective. Of course, since the fifth credential image sample is a distorted credential image sample, its authenticity is also good, so it can also be used to train a credential image classification model. Based on this, the first credential image sample, the second credential image sample, the fifth credential image sample and the sixth credential image sample can be used to train a credential image classification model for distinguishing the authenticity of credential images submitted by users. The solution in this embodiment is conducive to further improving the generation capability and type richness of credential image samples with good authenticity and offensive nature, and is convenient, fast and practical, thereby also helping to improve the classification accuracy of the trained credential image classification model.

[0137] In a feasible implementation manner, performing image correction processing on the initial credential image to obtain the user credential image may include:

[0138] The pose information of the initial voucher image is determined by using a pose estimation model based on scale-invariant feature transformation.

[0139] A second transformation matrix between the initial voucher image and the frontal perspective voucher image is determined according to the pose information of the initial voucher image.

[0140] Based on the second transformation matrix, an affine transformation process is performed on the initial credential image to obtain the user credential image.

[0141] In the embodiments of this specification, the principle of image correction processing for the initial credential image is consistent with the principle of image correction processing for the initial user credential image mentioned in the above embodiments, so it is not described in detail. In practical applications, the second transformation matrix between the initial credential image and the front view credential image is also equivalent to the affine transformation relationship between the initial user credential image and the user credential image. Therefore, the second transformation matrix can also be used to generate the fifth credential image sample later, which is not described in detail.

[0142] See also Figure 6, is a flow chart of a method for training a credential image classification model provided in an embodiment of this specification. The execution subject of the process may be an application for training the model, or a device equipped with an application for training the model. Figure 6 The process shown is described in detail, and the credential image classification model training method may specifically include the following steps:

[0143] Step 602, obtaining a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the above-mentioned credential image sample generation method and its embodiments.

[0144] In the embodiment of the present specification, when it is necessary to train a credential image classification model that can distinguish the authenticity of credential images, a training sample set including positive samples and negative samples can be obtained. The positive sample may include a trusted user credential image, and the trusted user credential image may refer to a user credential image obtained by collecting an image of the user's trusted credential. The negative sample may include a credential image obtained by using Figure 2 The offensive credential image samples generated by the scheme in the embodiment thereof, for example, one or more of the first credential image samples to the sixth credential image samples. In practical applications, the credential types to which the positive samples and negative samples contained in the training sample set belong can usually be consistent, which is conducive to ensuring the classification accuracy of the credential image classification model obtained by subsequent training.

[0145] Step 604: Use the training sample set to perform model training on the initial voucher image classification model to obtain a voucher image classification model.

[0146] In the embodiments of the present specification, the positive samples included in the training sample set may also be preset with first label data reflecting that they belong to a trusted credential image, and the negative samples included in the training sample set may also be preset with second label data reflecting that they do not belong to a trusted credential image. By using the training sample set carrying the corresponding label data to train the initial credential image classification model built using the deep learning model, the trained credential image classification model may be able to distinguish whether the target credential image of the user belongs to a trusted credential image.

[0147] Figure 6 The method in the paper can efficiently and conveniently obtain a large number of credential image samples with good authenticity and offensive nature, thereby meeting the training requirements for the credential image classification model, which is beneficial to improving the classification accuracy of the trained credential image classification model for whether the user credential image is credible, thereby ensuring the risk control effect.

[0148] See also Figure 7, is a flow chart of a method for classifying credential images provided in an embodiment of this specification. The execution subject of the process may be an application for classifying credential images, or a device equipped with the above application. Figure 7 The process shown is described in detail, and the credential image classification method may specifically include the following steps:

[0149] Step 702: Acquire the target credential image of the user.

[0150] Step 704, classify the target credential image using a credential image classification model to obtain a classification result that reflects whether the target credential image is a trusted credential image; wherein the credential image classification model is a model trained using the credential image classification model training method described above.

[0151] In the embodiment of the present specification, when it is necessary to identify whether the target credential image of the user is credible, a credential image classification model pre-trained using credential image samples consistent with the credential type to which the target credential image belongs can be obtained, so that the credential image classification model has the ability to distinguish the authenticity of the target credential image. Based on this, the target credential image can be input into the credential image classification model, thereby obtaining a classification result output by the credential image classification model to reflect whether the target credential image is a credible credential image.

[0152] Figure 7 In the method, since the scheme in the above embodiment can obtain a credential image classification model with good classification accuracy obtained by training with a large number of credential image samples with good authenticity and attack nature, the accuracy and effectiveness of the classification result generated by the credential image classification model to reflect whether the user's target credential image belongs to a trusted credential image can be guaranteed, which is conducive to ensuring the risk control effect.

[0153] Based on the same idea, the present specification also provides the above Figure 2 The device corresponding to the method in . See Figure 8 , is a schematic diagram of the structure of a credential image sample generating device provided in an embodiment of this specification. Figure 8 As shown, the credential image sample generating device 8 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the credential image sample generating device 8 may include an acquisition module 81, a determination module 82, an image processing module 83 and a sample generating module 84, wherein:

[0154] The acquisition module 81 is used to acquire the user credential image.

[0155] The determination module 82 is configured to determine an initial target object image included in the user credential image.

[0156] The image processing module 83 is used to use a generative artificial intelligence model to perform image rendering processing on the target object area or the background area in the initial target object image to obtain a fictitious target object image.

[0157] The sample generating module 84 is configured to replace the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample.

[0158] Optionally, the generative artificial intelligence model includes an image restoration model.

[0159] Correspondingly, the image processing module 83 may include:

[0160] The first segmentation unit is used to perform image segmentation processing on the initial target object image by using an image segmentation model to obtain a mask image used to mask the area where the target object in the initial target object image is located.

[0161] The background restoration unit is used to perform background area restoration processing based on the initial target object image and the mask image by using the image restoration model to obtain a background restoration image.

[0162] The first generating unit is configured to generate a fictitious target object image according to the background restoration image and the image in the region where the target object is located in the initial target object image.

[0163] Optionally, the user credential image may include: a trusted user credential image and an untrusted user credential image.

[0164] Correspondingly, the first generating unit may be specifically used for:

[0165] Generate a fictitious target object image based on the background repair image generated based on any of the untrusted user credential images and an image in the area where the target object is located in any of the trusted user credential images; and / or,

[0166] A fictitious target object image is generated according to the background repair image generated based on any one of the trusted user credential images and an image in the area where the target object is located in any one of the untrusted user credential images.

[0167] Correspondingly, the first generating unit may be specifically used for:

[0168] Performing image stitching processing on the background restoration image and the image in the area where the target object is located in the initial target object image to obtain a fictitious target object image; or,

[0169] An image fusion process is performed on the background restoration image and the image in the region where the target object is located in the initial target object image to obtain a fictitious target object image.

[0170] Optionally, the generative artificial intelligence model may include a diffusion model;

[0171] Correspondingly, the image processing module 83 may include:

[0172] A second segmentation unit, configured to perform image segmentation processing on the initial target object image using an image segmentation model to obtain a mask image for masking a region where the target object is located in the initial target object image;

[0173] The second generation unit is used to use the diffusion model to process based on the initial target object image, the mask image and the preset prompt word to obtain a fictitious target object image output by the diffusion model; wherein the preset prompt word is used to guide the diffusion model to generate a target object image that conforms to a preset style in the area where the target object is located.

[0174] Optionally, the sample generation module 84 may include:

[0175] A position determination unit is used to determine the position information of the initial target object image in the user credential image.

[0176] A first sample generating unit is configured to perform image stitching processing on the fictitious target object image and the user credential image based on the position information of the initial target object image to obtain a first credential image sample; wherein the position of the fictitious target object image in the first credential image sample is consistent with the position of the initial target object image in the user credential image; or

[0177] and a second sample generating unit, configured to perform image fusion processing on the fictitious target object image and the user credential image based on the position information of the initial target object image to obtain a second credential image sample; wherein the position of the fictitious target object image in the second credential image sample is consistent with the position of the initial target object image in the user credential image.

[0178] Optionally, the acquisition module 81 may include:

[0179] The first acquisition unit is used to acquire an initial user credential image.

[0180] The first correction unit is used to perform image correction processing on the initial user credential image to obtain a corrected image.

[0181] The first detection unit is used to perform credential corner point detection processing on the corrected image to obtain the user credential image contained in the corrected image.

[0182] The sample generation module 84 may further include:

[0183] A third sample generating unit is configured to replace the user credential image contained in the corrected image with the first credential image sample or the second credential image sample to obtain a third credential image sample; and / or,

[0184] The fourth sample generating unit is configured to perform affine transformation processing on the third credential image sample according to the affine transformation relationship between the initial user credential image and the corrected image, so as to obtain a fourth credential image sample having a posture consistent with that of the initial user credential image.

[0185] Optionally, the first correction unit may be specifically configured to:

[0186] The pose information of the initial user credential image is determined by using a pose estimation model based on scale-invariant feature transformation.

[0187] A first transformation matrix between the initial user credential image and the frontal perspective credential image is determined according to the pose information of the initial user credential image.

[0188] Based on the first transformation matrix, an affine transformation process is performed on the initial user credential image to obtain the corrected image.

[0189] Optionally, the acquisition module 81 may include:

[0190] The second acquiring unit is used to acquire an initial user credential image.

[0191] The second detection unit is used to perform credential corner point detection processing on the initial user credential image to obtain an initial credential image.

[0192] The second correction unit is used to perform image correction processing on the initial credential image to obtain the user credential image.

[0193] The sample generation module 84 may further include:

[0194] a fifth sample generating unit, configured to perform affine transformation processing on the first credential image sample or the second credential image sample according to an affine transformation relationship between the initial user credential image and the user credential image, so as to obtain a fifth credential image sample having a posture consistent with that of the initial credential image; and / or,

[0195] The sixth sample generating unit is configured to replace the initial credential image included in the initial user credential image with the fifth credential image sample to obtain a sixth credential image sample.

[0196] Optionally, the second correction unit may be specifically used to:

[0197] The pose information of the initial voucher image is determined by using a pose estimation model based on scale-invariant feature transformation.

[0198] A second transformation matrix between the initial voucher image and the frontal perspective voucher image is determined according to the pose information of the initial voucher image.

[0199] Based on the second transformation matrix, an affine transformation process is performed on the initial credential image to obtain the user credential image.

[0200] Optionally, the target object contained in the target object area in the initial target object image may include at least one of a credential anti-counterfeiting object, a user biometric feature object, and a physical object certified by a credential.

[0201] Based on the same idea, the present specification also provides the above Figure 6 The device corresponding to the method in . See Fig. 9 , is a schematic diagram of the structure of a credential image classification model training device provided in an embodiment of this specification. Fig. 9 As shown, the voucher image classification model training device 9 can be implemented as all or part of the electronic device through software, hardware or a combination of both. According to some embodiments, the voucher image classification model training device 9 may include an acquisition module 91 and a training module 92, wherein:

[0202] The acquisition module 91 is used to acquire a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the credential image sample generation method described above.

[0203] The training module 92 is used to perform model training on the initial voucher image classification model using the training sample set to obtain the voucher image classification model.

[0204] Based on the same idea, the present specification also provides the above Figure 7The device corresponding to the method in . See Fig.10 , is a schematic diagram of the structure of a voucher image classification device provided in an embodiment of this specification. Fig.10 As shown, the credential image classification device 10 can be implemented as all or part of an electronic device through software, hardware or a combination of both. According to some embodiments, the credential image classification device 10 may include an acquisition module 101 and a classification module 102, wherein:

[0205] The acquisition module 101 is used to acquire a target credential image of a user.

[0206] The classification module 102 is used to classify the target credential image using a credential image classification model to obtain a classification result reflecting whether the target credential image is a credential image; wherein the credential image classification model is a model trained by the above-mentioned credential image classification model training method.

[0207] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0208] The embodiments of this specification also provide a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the credential image sample generation method, the credential image classification model training method, and the credential image classification method as described above are implemented. The specific execution process can be found in the specific description of the relevant embodiments of the credential image sample generation method, which will not be repeated here.

[0209] The present specification also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor, such as the above-mentioned credential image sample generation method, credential image classification model training method and credential image classification method. The specific execution process can be referred to the specific description of the relevant embodiments of the credential image sample generation method, which will not be repeated here.

[0210] The embodiments of this specification also provide Fig.11 The structural diagram of the electronic device shown in FIG. Fig.11At the hardware level, the electronic device may include a processor, an internal bus, a network interface, a memory, and a non-volatile memory, and may also include hardware required for other services. The processor reads the corresponding computer program from the non-volatile memory into the memory and then runs it to implement the above-mentioned voucher image sample generation method, voucher image classification model training method, and voucher image classification method. The specific execution process can refer to the specific description of the relevant embodiments of the voucher image sample generation method, which will not be repeated here.

[0211] Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the executor of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0212] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for computer-readable storage media, computer program products and Fig.11 As for the electronic device shown, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0213] In the 1990s, improvements to a technology could be clearly distinguished as hardware improvements (for example, improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the method flow). However, with the development of technology, many improvements to the method flow today can be regarded as direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved method flow into the hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (such as a field programmable gate array (FPGA)) is such an integrated circuit whose logical function is determined by the user's programming of the device. Designers can "integrate" a digital system on a PLD by programming it themselves, without having to ask a chip manufacturer to design and produce a dedicated integrated circuit chip. Moreover, nowadays, instead of manually making integrated circuit chips, this kind of programming is mostly implemented by "logic compiler" software, which is similar to the software compiler used when developing and writing programs, and the original code before compilation must also be written in a specific programming language, which is called hardware description language (HDL). There is not only one HDL, but many kinds, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also know that it is only necessary to program the method flow slightly in the above-mentioned hardware description languages ​​and program it into the integrated circuit, and then it is easy to obtain the hardware circuit that implements the logic method flow.

[0214] The controller can be implemented in any appropriate manner, for example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing a computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, a logic gate, a switch, an application-specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that in addition to implementing the controller in a purely computer-readable program code manner, the controller can be implemented in the form of a logic gate, a switch, an application-specific integrated circuit, a programmable logic controller, and an embedded microcontroller by logically programming the method steps. Therefore, this controller can be considered as a hardware component, and the devices included therein for implementing various functions can also be regarded as structures within the hardware component. Or even, the devices for implementing various functions can be regarded as both software modules for implementing the method and structures within the hardware component.

[0215] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0216] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0217] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0218] This specification is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of this specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0219] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0220] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0221] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0222] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0223] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0224] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0225] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, this specification may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0226] This specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. This specification may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0227] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0228] The above description is only an embodiment of the present specification and is not intended to limit the present specification. For those skilled in the art, the present specification may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of the claims of the present specification.

Claims

1. A method for generating a voucher image sample, the method comprising: Get the user credential image; determining an initial target object image contained in the user credential image; Using a generative artificial intelligence model, image rendering is performed on a target object region or a background region in the initial target object image to obtain a fictitious target object image; The initial target object image in the user credential image is replaced by the fictitious target object image to obtain a credential image sample.

2. The method according to claim 1, wherein the generative artificial intelligence model comprises an image restoration model; The method of using a generative artificial intelligence model to perform image rendering processing on a background area in the initial target object image to obtain a fictitious target object image includes: Performing image segmentation processing on the initial target object image using an image segmentation model to obtain a mask image used to mask the area where the target object is located in the initial target object image; Using the image restoration model, a background region restoration process is performed based on the initial target object image and the mask image to obtain a background restoration image; A fictitious target object image is generated according to the background restoration image and an image in a region where the target object is located in the initial target object image.

3. The method of claim 2, wherein the user credential image comprises: Trusted user credential images and untrusted user credential images; The generating of a fictitious target object image according to the background restoration image and the image in the region where the target object is located in the initial target object image comprises: generating a fictitious target object image according to the background repair image generated based on any one of the untrusted user credential images and an image in a region where the target object is located in any one of the trusted user credential images; and / or, A fictitious target object image is generated according to the background repair image generated based on any one of the trusted user credential images and an image in the area where the target object is located in any one of the untrusted user credential images.

4. The method according to claim 2, wherein generating a fictitious target object image based on the background restoration image and the image in the region where the target object is located in the initial target object image comprises: Performing image stitching processing on the background restoration image and the image in the area where the target object is located in the initial target object image to obtain a fictitious target object image; or, An image fusion process is performed on the background restoration image and the image in the region where the target object is located in the initial target object image to obtain a fictitious target object image.

5. The method of claim 1, wherein the generative artificial intelligence model comprises a diffusion model; The method of using a generative artificial intelligence model to perform image rendering processing on the target object region in the initial target object image to obtain a fictitious target object image includes: Performing image segmentation processing on the initial target object image using an image segmentation model to obtain a mask image used to mask the area where the target object is located in the initial target object image; The diffusion model is used to process the initial target object image, the mask image and the preset prompt words to obtain a fictitious target object image output by the diffusion model; wherein the preset prompt words are used to guide the diffusion model to generate a target object image that conforms to a preset style in the area where the target object is located.

6. The method according to claim 1, wherein replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample comprises: determining location information of the initial target object image in the user credential image; Based on the position information of the initial target object image, image stitching processing is performed on the fictitious target object image and the user credential image to obtain a first credential image sample; wherein the position of the fictitious target object image in the first credential image sample is consistent with the position of the initial target object image in the user credential image; or, Based on the position information of the initial target object image, image fusion processing is performed on the fictitious target object image and the user credential image to obtain a second credential image sample; wherein the position of the fictitious target object image in the second credential image sample is consistent with the position of the initial target object image in the user credential image.

7. The method according to claim 6, wherein obtaining the user credential image comprises: Get the initial user credential image; Performing image correction processing on the initial user credential image to obtain a corrected image; Performing credential corner point detection processing on the corrected image to obtain the user credential image contained in the corrected image; The step of replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample further includes: replacing the user credential image contained in the corrected image with the first credential image sample or the second credential image sample to obtain a third credential image sample; and / or, According to the affine transformation relationship between the initial user credential image and the corrected image, an affine transformation process is performed on the third credential image sample to obtain a fourth credential image sample having a posture consistent with that of the initial user credential image.

8. The method according to claim 7, wherein the performing image correction processing on the initial user credential image to obtain a corrected image comprises: Determining the pose information of the initial user credential image by using a pose estimation model based on scale-invariant feature transformation; Determining a first transformation matrix between the initial user credential image and the front view credential image according to the pose information of the initial user credential image; Based on the first transformation matrix, an affine transformation process is performed on the initial user credential image to obtain the corrected image.

9. The method according to claim 6, wherein obtaining the user credential image comprises: Get the initial user credential image; Performing credential corner point detection processing on the initial user credential image to obtain an initial credential image; Performing image correction processing on the initial credential image to obtain the user credential image; The step of replacing the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample further includes: According to the affine transformation relationship between the initial user credential image and the user credential image, performing affine transformation processing on the first credential image sample or the second credential image sample to obtain a fifth credential image sample having a posture consistent with the initial credential image; and / or, The initial credential image included in the initial user credential image is replaced with the fifth credential image sample to obtain a sixth credential image sample.

10. The method according to claim 9, wherein the performing image correction processing on the initial credential image to obtain the user credential image comprises: Determining the pose information of the initial voucher image by using a pose estimation model based on scale-invariant feature transformation; Determining a second transformation matrix between the initial voucher image and the frontal perspective voucher image according to the pose information of the initial voucher image; Based on the second transformation matrix, an affine transformation process is performed on the initial credential image to obtain the user credential image.

11. The method according to any one of claims 1 to 10, wherein the target object contained in the target object region in the initial target object image comprises: At least one of a credential anti-counterfeiting object, a user biometric object, and an entity object certified by the credential.

12. A voucher image classification model training method, comprising: Acquire a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the credential image sample generation method according to any one of claims 1 to 11; The training sample set is used to perform model training on the initial voucher image classification model to obtain the voucher image classification model.

13. A voucher image classification method, comprising: Get the target credential image of the user; The target credential image is classified using a credential image classification model to obtain a classification result that reflects whether the target credential image is a trusted credential image; wherein the credential image classification model is a model trained using the credential image classification model training method described in claim 12.

14. A voucher image sample generating device, comprising: An acquisition module, used to acquire a user credential image; a determination module, configured to determine an initial target object image contained in the user credential image; An image processing module, used to use a generative artificial intelligence model to perform image rendering processing on a target object area or a background area in the initial target object image to obtain a fictitious target object image; The sample generation module is used to replace the initial target object image in the user credential image with the fictitious target object image to obtain a credential image sample.

15. The apparatus of claim 14, wherein the generative artificial intelligence model comprises an image restoration model; The image processing module comprises: A first segmentation unit, configured to perform image segmentation processing on the initial target object image using an image segmentation model to obtain a mask image for masking a region where the target object is located in the initial target object image; A background restoration unit, configured to perform background region restoration processing based on the initial target object image and the mask image using the image restoration model to obtain a background restoration image; The first generating unit is configured to generate a fictitious target object image according to the background restoration image and the image in the region where the target object is located in the initial target object image.

16. The apparatus of claim 14, wherein the generative artificial intelligence model comprises a diffusion model; The image processing module comprises: A second segmentation unit, configured to perform image segmentation processing on the initial target object image using an image segmentation model to obtain a mask image for masking a region where the target object is located in the initial target object image; The second generation unit is used to use the diffusion model to process based on the initial target object image, the mask image and the preset prompt word to obtain a fictitious target object image output by the diffusion model; wherein the preset prompt word is used to guide the diffusion model to generate a target object image that conforms to a preset style in the area where the target object is located.

17. A voucher image classification model training device, comprising: An acquisition module, configured to acquire a training sample set; wherein the positive samples in the training sample set include trusted user credential images, and the negative samples in the training sample set include the credential image samples generated by the credential image sample generation method according to any one of claims 1 to 11; The training module is used to perform model training on the initial voucher image classification model using the training sample set to obtain the voucher image classification model.

18. A voucher image classification device, comprising: An acquisition module, used to acquire a target credential image of a user; A classification module is used to classify the target credential image using a credential image classification model to obtain a classification result that reflects whether the target credential image is a trusted credential image; wherein the credential image classification model is a model trained using the credential image classification model training method described in claim 12.

19. A computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 13.

20. An electronic device, comprising: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the steps of the method as claimed in any one of claims 1 to 13.

21. A computer program product having at least one instruction stored thereon, wherein when the at least one instruction is executed by a processor, the steps of the method according to any one of claims 1 to 13 are implemented.

Images