Method and device for generating samples and training image recognition network and card and certificate classification network

Abstract

The embodiment of the invention provides a sample generation method and device, an image recognition network training method and device and a card classification network training method and device, and the method comprises the steps: obtaining an image template which comprises a plurality of layers which are used for displaying the information of a specified type; obtaining first information used for generating an image, wherein the first information comprises information of a specified type corresponding to each image layer; generating a first generated image based on the image template and the first information; a plurality of second generated images are generated based on the first generated image through the style conversion network, the second generated images have different environment backgrounds relative to the first generated image, the second generated images serve as negative samples for training an image recognition network, and the image recognition network is used for recognizing whether the input image is a real image or not; therefore, more diverse negative samples for training the image recognition network are generated.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and more particularly to a method and device for generating samples, training image recognition networks, and training card classification networks. Background Art

[0002] With the development and popularization of AIGC (Artificial Intelligence Generated Content) technology, a large number of forged images generated based on AIGC have emerged. In the network environment, people often cannot distinguish which are real images and which are forged images generated based on AIGC from a large number of network images.

[0003] However, in some scenarios, forged images generated based on AIGC may cause trouble to people's lives. Therefore, it is intuitively important to provide an image recognition network for distinguishing the authenticity of images. In order to ensure that the trained image recognition network can well identify and distinguish the authenticity of the input images, it is necessary to generate a sufficient number of forged images and use them as negative samples to train the image recognition network. Therefore, how to provide a sample generation method to obtain negative samples for training the image recognition network has become an urgent problem to be solved. Summary of the invention

[0004] One or more embodiments of the present specification provide a method and apparatus for sample generation, training an image recognition network, and training a card classification network, so as to generate a variety of generated images, and then use the generated images as negative samples to train an image recognition network that can recognize the generated images.

[0005] According to a first aspect, a sample generation method is provided, comprising:

[0006] Acquire an image template, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type;

[0007] Acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0008] Based on the image template and the first information, generate a first generated image;

[0009] Based on the first generated image, a number of second generated images are generated through a style conversion network, wherein the second generated images have different environmental backgrounds relative to the first generated image, and the second generated images are used as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image.

[0010] According to a second aspect, there is provided a method for training an image recognition network, comprising:

[0011] Acquire an image template, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type;

[0012] Acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0013] Based on the image template and the first information, generate a first generated image;

[0014] generating, based on the first generated image, a plurality of second generated images through a style conversion network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image;

[0015] Based on the plurality of second generated images, an image recognition network is trained, wherein the image recognition network is used to identify whether an input image is a real image.

[0016] According to a third aspect, a method for training a card classification network is provided, comprising:

[0017] Obtaining a card template corresponding to the first card, wherein the card template includes multiple layers, and the multiple layers are used to display user information;

[0018] Acquire first user information for generating a card image;

[0019] Generate a first card image based on the card template and the first user information;

[0020] generating a plurality of second card images based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image;

[0021] Based on the plurality of second card images, a card classification network is trained, wherein the card classification network is used to identify whether an input card image is a real card image.

[0022] According to a fourth aspect, there is provided a sample generating device, comprising:

[0023] A first acquisition module is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type;

[0024] A second acquisition module is configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0025] A first generating module, configured to generate a first generated image based on the image template and the first information;

[0026] The second generation module is configured to generate a plurality of second generated images based on the first generated image through a style transfer network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image, and the plurality of second generated images serve as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image.

[0027] According to a fifth aspect, there is provided a device for training an image recognition network, comprising:

[0028] A third acquisition module is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type;

[0029] a fourth acquisition module, configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0030] A third generating module, configured to generate a first generated image based on the image template and the first information;

[0031] a fourth generating module, configured to generate a plurality of second generated images based on the first generated image through a style conversion network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image;

[0032] The first training module is configured to train an image recognition network based on a plurality of second generated images, wherein the image recognition network is used to identify whether an input image is a real image.

[0033] According to a sixth aspect, there is provided a device for training a card classification network, comprising:

[0034] a fifth acquisition module, configured to acquire a card template corresponding to the first card, wherein the card template includes a plurality of layers, and the plurality of layers are used to display user information;

[0035] A sixth acquisition module, configured to acquire first user information for generating a card image;

[0036] a fifth generating module, configured to generate a first card image based on the card template and the first user information;

[0037] a sixth generation module, configured to generate a plurality of second card images based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image;

[0038] The second training module is configured to train a card classification network based on the plurality of second card images, wherein the card classification network is used to identify whether an input card image is a real card image.

[0039] According to a seventh aspect, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed in a computer, the computer is caused to execute the method described in the first aspect, the second aspect or the third aspect.

[0040] According to an eighth aspect, a computing device is provided, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method described in the first aspect, the second aspect or the third aspect is implemented.

[0041] According to the method and device for sample generation, training image recognition network and training card classification network provided in the embodiments of this specification, an image template is obtained, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type; first information for generating an image is obtained, and the first information includes information of a specified type corresponding to each layer; a first generated image is generated based on the image template and the first information; and a plurality of second generated images are generated based on the first generated image through a style transfer network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image, and the plurality of second generated images are used as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image, so as to generate more diverse negative samples for training the image recognition network, and then train an image recognition network that can recognize the generated images. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0043] Figure 1 A schematic diagram of an implementation framework of an embodiment disclosed in this specification;

[0044] Figure 2 A schematic diagram of a flow chart of a sample generation method provided in an embodiment;

[0045] Figure 3A A schematic diagram of a first image provided by an embodiment;

[0046] Figure 3BA schematic diagram of a first fused image fused with a background provided in an embodiment;

[0047] Figure 4 A schematic diagram of a flow chart of a method for training an image recognition network provided in an embodiment;

[0048] Figure 5 A flowchart of a method for training a card classification network provided in an embodiment;

[0049] Figure 6 A schematic block diagram of a sample generation device provided in an embodiment;

[0050] Figure 7 A schematic block diagram of a device for training an image recognition network provided in an embodiment;

[0051] Figure 8 A schematic block diagram of a device for training a card classification network provided in an embodiment. DETAILED DESCRIPTION

[0052] The technical solutions of the embodiments of this specification will be described in detail below with reference to the accompanying drawings.

[0053] As mentioned above, in order to ensure that the trained image recognition network can well identify and distinguish the authenticity of the input image, it is necessary to generate a sufficient number of forged images as negative samples to train the image recognition network. Therefore, how to provide a sample generation method to obtain negative samples for training the image recognition network becomes an urgent problem to be solved.

[0054] In view of this, the inventor proposes a sample generation method. Figure 1 A schematic diagram of an implementation scenario according to an embodiment disclosed in this specification is shown. In this implementation scenario, an image template is first obtained, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type; then, first information for generating an image is obtained, and the first information includes information of a specified type corresponding to each layer; based on the image template and the first information, a first generated image is generated, and the multiple layers in the image template can indicate the display position of each information in the first information in its corresponding layer (i.e., the display position in the first generated image), and accordingly, each corresponding information of the specified type in the first information is displayed at each designated display position in the first generated image; then, the first generated image is processed through a style conversion network to transform the environmental background in the first generated image, thereby generating a number of second generated images whose environmental backgrounds are different from the environmental background of the first generated image. In addition, the environmental backgrounds between the several second generated images are also different, and the several second generated images are used as negative samples for training an image network, and the image network is used to identify whether the input image is a real image.

[0055] In the above process, a first generated image can be generated through an image template and the first information, and then the first generated image can be processed through a style transfer network to transform the environmental background of the first generated image, thereby generating a number of second generated images with different environmental backgrounds from the first generated image, thereby achieving the generation of more diverse generated images based on artificial intelligence technology, and then using a number of second generated images (i.e., forged images generated based on artificial intelligence technology) as negative samples for training an image recognition network to train the image recognition network, thereby helping to improve the image recognition network's ability to recognize generated images (i.e., forged images generated based on artificial intelligence technology), thereby protecting the network environment, helping users distinguish the authenticity of images in the network environment, and improving user experience.

[0056] In the following, in conjunction with specific embodiments, the sample generation method and device, the method and device for training an image recognition network, and the method and device for training a card classification network provided in this specification are described in detail.

[0057] It can be understood that in the process of training an image recognition network for identifying whether an input image is a real image, generated images (generated fake images) are needed as negative samples and real images (images actually captured by an image acquisition device) are needed as positive samples. The image recognition network is trained through the aforementioned positive and negative samples so that the image recognition network can learn the characteristic information of the real image and the generated image, so as to identify the authenticity of the image and realize the classification and recognition of true and false images.

[0058] In order to generate more diverse and sufficient generated images for training image recognition networks, the embodiments of this specification provide a process for generating samples (i.e., images) based on artificial intelligence technology, that is, through a style conversion network, the generated images (i.e., forged images) are processed to transform the specified style (i.e., environmental background) of the generated images to obtain more diverse images, so as to obtain negative samples for training image recognition networks. The style conversion network can be any neural network that can transform the specified style in the image.

[0059] In some possible examples, the style transfer network can be a generative network in a generative adversarial network, which can transform the image into a specified style by leveraging the ability of this type of network to reconstruct the image based on the encoded information in the latent space. For example, based on the generated image, its corresponding encoded information in the latent space is obtained, and then the information corresponding to the specified style in the encoded information is adjusted to obtain new encoded information. Based on the new encoded information, a new generated image after the specified style is transformed is generated through the style transfer network to obtain more diverse generated images (i.e., negative samples for training image recognition networks).

[0060] In order to enable the style transfer network (i.e., the generative network in the generative adversarial network) to have the ability to generate images, it is necessary to train the generative adversarial network using images (including real images and generated images) and their corresponding first label data indicating whether they are real images, and then use the generative network in the trained generative adversarial network as the style transfer network in the present exemplary embodiment.

[0061] It can be understood that the generative adversarial network includes a generative network G and a discriminative network D, wherein the generative network G is used to reconstruct the image based on the encoded information in the latent space, and the discriminative network D is used to determine whether the image input to it is a real image (that is, to determine whether the image input to it is an image generated by the generative network G).

[0062] In the process of training the generative adversarial network, the training of the generative network G and the training of the discriminative network D can be performed alternately. In some specific examples, the parameters of the generative network G can be fixed first, and the discriminative network D can be trained using the real image (first sample image) and its corresponding first label data indicating that it is a real image, and the generated image (first sample image) generated by the generative network G and its corresponding first label data indicating that it is a fake image, until the discriminative network D reaches the first convergence condition. After that, the parameters of the discriminative network D are fixed, and the generative network G is trained using the output result of the discriminative network D. The real image is an image obtained by actually shooting the sample subject. In some examples, the sample subject can be, for example, but not limited to, a card, or other subjects, such as an object, a book, a building, and a bill. Exemplarily, in the case where the image to be generated is a card image, the sample subject can be a card; in the case where the image to be generated is a bill image, the sample subject can be a bill.

[0063] In some examples, the aforementioned first sample image may include the imaging content of the sample body. In other examples, the aforementioned first sample image may also include both the imaging content of the sample body and the imaging content of the background (such as plants, desktops, walls or other backgrounds) in which the sample body is located.

[0064] Next, the process of training the discriminant network is first introduced, which may include: determining the discriminant network D to be trained in the generative adversarial network, obtaining the first sample image X1 and its corresponding first label data Y1 indicating whether it is a real image, wherein, when the first sample image X1 is a real image, its corresponding first label data Y1 is data indicating that it is a real image, and when the first sample image X1 is an image generated by the generative network G, its corresponding first label data Y1 is data indicating that it is a false (i.e., forged) image; inputting the first sample image X1 into the discriminant network D, and obtaining the first result data outputted by the discriminant network D for determining whether the first sample image X1 is a real image; determining the first loss based on the difference between the first result data and the first label data Y1; training the discriminant network D with the goal of minimizing the first loss; performing the above process multiple times until the discriminant network D reaches the first convergence condition, and obtaining the discriminant network D at this time.

[0065] In some possible examples, after acquiring the above-mentioned first sample image, the first sample image can be preprocessed. The preprocessing may include but is not limited to at least one of the following processes: cropping, scaling, and preset data enhancement (such as adaptively adjusting image clarity, etc.), so that the size of the preprocessed image meets the input requirements of the discriminant network.

[0066] Among them, the first convergence condition may include but is not limited to the following conditions: the number of iterative training of the discriminant network reaches a first number threshold, the duration of training the discriminant network reaches a first duration threshold; the first loss is lower than the first loss threshold.

[0067] Next, the training process of the generative network G in the generative adversarial network, i.e., the style transfer network, is introduced. The process of training the style transfer network includes the following steps 11-15:

[0068] In step 11, a style transfer network to be trained in a generative adversarial network is obtained, wherein the generative adversarial network also includes the aforementioned discriminant network D, which has been trained based on each first sample image and its corresponding first label data indicating whether it is a real image. Each first sample image includes the aforementioned real image and an image generated by the generative network G.

[0069] In step 12, a third vector in the latent space is obtained. In some possible examples, the third vector may be obtained by random sampling from a predefined probability distribution (e.g., a Gaussian distribution). In some other possible examples, the third vector may also be a vector obtained by performing a specified adjustment on a vector randomly sampled from a predefined probability distribution (e.g., a Gaussian distribution). The third vector includes the encoded information in the latent space.

[0070] Then, in step 13, the current generated image is obtained based on the third vector through the style conversion network to be trained. In this step, the third vector is input into the style conversion network to be trained so that the style conversion network processes the third vector to obtain the current generated image.

[0071] Next, in step 14, a discrimination result of whether the current generated image is a real image is obtained based on the current generated image through the discriminant network D. In this step, the current generated image is input into the discriminant network D so that the discriminant network D processes the current generated image to obtain a discrimination result of whether the current generated image is a real image.

[0072] Then, in step 15, based on the aforementioned discrimination result, the parameters of the style transfer network to be trained are adjusted to obtain a trained style transfer network. In this step, based on the difference between the aforementioned discrimination result and the first label data indicating that the currently generated image is a fake image, a second loss is determined, and the parameters of the style transfer network to be trained (i.e., the aforementioned generating network G) are adjusted with the goal of minimizing the second loss.

[0073] It is understandable that the aforementioned steps 12-15 may be iteratively performed multiple times until the style transfer network to be trained reaches the second convergence condition to obtain a trained style transfer network. The second convergence condition may include but is not limited to the following conditions: the number of iterative training of the discriminant network reaches the second number threshold, the duration of training the style transfer network reaches the second duration threshold; the second loss is lower than the second loss threshold.

[0074] In some possible examples, after obtaining a trained style transfer network through the above training process, the parameters of the trained style transfer network can be fixed again, and the discriminant network can be trained again based on the above training process of the discriminant network, and so on, the discriminant network and the style transfer network (i.e., the generative network) are alternately trained until a preset balance condition is reached between the discriminant network and the style transfer network, and it is determined that the training of the generative adversarial network is completed to obtain the style transfer network. The preset balance condition may include but is not limited to: the quality of the image generated by the style transfer network based on the encoded information in the latent space reaches a preset quality threshold; the change of the corresponding losses of the discriminant network and the style transfer network in multiple iterations does not exceed a preset value, etc.

[0075] In some possible examples, the aforementioned generative adversarial network may be a StyleGAN2 network, a StyleGAN3 network, or a GAN-inversion to ensure that a card image with better quality is generated.

[0076] At this time, the trained style transfer network can reconstruct the image based on the encoded information in the latent space (such as the aforementioned third vector). It can be understood that in the encoded information in the latent space, the information at different positions can control the different styles of the generated image, which includes, but is not limited to, the lighting display style in the generated image, and the background style in the generated image (for example, the background of the sample body in the generated image is a desktop, the background of the sample body in the generated image is a wall, and for example: the background of the sample body in the generated image is a glass desktop, the background of the sample body in the generated image is a white solid wood desktop, etc.).

[0077] In view of the above situation, by controlling the information corresponding to the specified style in the encoding information in the latent space, the control of the specified style of the image generated by the style transfer network can be achieved. Referring to the above ideas, the sample generation method provided in the embodiment of this specification can draw on the ability of the style transfer network to reconstruct images based on the encoding information in the latent space, and obtain more diverse encoding information (which represents different environmental backgrounds) by modifying the information corresponding to the specified style (such as environmental background) in the encoding information obtained according to the generated image (forged image). Based on the more diverse encoding information, more generated images with different environmental backgrounds can be generated to improve the diversity of the generated images and better improve the number of generated images, providing a basis for the subsequent training of the image recognition network.

[0078] In some other possible examples, the style transfer network can be a pre-trained style transfer network based on MimicBrush, which is an artificial intelligence image editing framework. The input of this type of style transfer network includes the source image of the occluded area (including the source image and the mask used to occlude the partial area of ​​the source image, which can also be called a mask) and the reference image. The style transfer network can imitate the information in the reference image to realize the editing and generation of the content of the occluded area in the source image, so as to replace the content of the occluded area in the source image with the content in the reference image. In the embodiments of this specification, the ability of this type of network is used to transform the background in the generated image fused with the background to obtain more diverse card images.

[0079] In some possible examples, the current pre-trained MimicBrush-based network can be directly used as a style transfer network. In some other possible examples, in order to generate a generated image with better quality, the embodiment of this specification also provides a training process for this type of style transfer network. The training process of this type of style transfer network is introduced below. Specifically, the training process may include steps 21-24:

[0080] In step 21, a second sample image and a corresponding fourth background image are obtained, wherein the second sample image is an image obtained by photographing the sample subject and the background in which it is located. In this step, a training sample set may be pre-constructed, and the training sample set includes a plurality of pairs of sample images, each pair of sample images includes a frame of sample image (referred to as the second sample image) and a corresponding frame of background image (referred to as the fourth background image), and the second sample image is an image obtained by photographing the corresponding sample subject and the background in which it is located. In step 21, any pair of sample images may be obtained from the training sample set to obtain the second sample image and the corresponding fourth background image.

[0081] Then, in step 22, the second sample image is subjected to occlusion processing to occlude the background area in the second sample image. In this step, the second sample image can be subjected to occlusion processing (i.e., mask processing, for example, creating a mask corresponding to the second sample image) to occlude the background area in the second sample image (i.e., occluding the other areas in the second sample image except the area where the sample body is located). The occluded background area is the area in the second sample image that needs to be replaced.

[0082] Next, in step 23, the fourth background image and the second sample image after occlusion processing are input into the style transfer network to be trained, so as to replace the content of the background area of ​​the second sample image after occlusion processing with the content in the fourth background image to obtain an output image.

[0083] In some examples, the style transfer network to be trained may include an encoding subnetwork (e.g., a first encoding subnetwork and a second encoding subnetwork) and two U-Net subnetworks, namely, an imitation U-Net subnetwork and a reference U-Net subnetwork.

[0084] Among them, the imitation U-Net subnetwork and the reference U-Net subnetwork both include multiple layers of encoding layers and multiple layers of decoding layers, wherein each decoding layer can be a decoding layer based on the attention mechanism, and the imitation U-Net subnetwork and the reference U-Net subnetwork share the key matrix and value matrix in their corresponding decoding layers, that is, the key matrix and value matrix extracted from the input features of each decoding layer of the reference U-Net subnetwork are injected into the corresponding decoding layer of the imitation U-Net subnetwork to fuse the features of the fourth background image with the features of the second sample image after occlusion processing, thereby helping the imitation U-Net subnetwork to more accurately complete the generation of the content of the occluded area in the second sample image based on the features of the fourth background image.

[0085] Hereinafter, the encoding layer and the decoding layer in the reference U-Net sub-network are referred to as the first encoding layer and the first decoding layer, respectively, and the encoding layer and the decoding layer in the imitation U-Net sub-network are referred to as the second encoding layer and the second decoding layer, respectively.

[0086] In step 23, the first encoding sub-network can be used to obtain a potential representation corresponding to the fourth background image (referred to as the first potential representation) based on the fourth background image, and the first potential representation is used as the input of the reference U-Net sub-network. The second encoding sub-network processes the second sample image after occlusion processing (i.e., the second sample image and its corresponding mask) to obtain a potential representation corresponding to the second sample image (referred to as the second potential representation) and a potential representation of the unoccluded area in the second sample image (referred to as the third potential representation), and the second potential representation, the third potential representation and the aforementioned mask are used as the input of the imitation U-Net sub-network.

[0087] Then, the first latent representation is input into the reference U-Net sub-network, and the second latent representation, the third latent representation, and the aforementioned mask are input into the imitation U-Net sub-network, so as to jointly process their corresponding inputs through the reference U-Net sub-network and the imitation U-Net sub-network to generate the content of the occluded area in the second sample image based on the fourth background image, and obtain an output image.

[0088] The aforementioned process of obtaining the output image may include: extracting features from the input (third potential representation) of the reference U-Net subnetwork through multiple first encoding layers to obtain the first features corresponding to the fourth background image; sequentially using the first decoding layers of the reference U-Net subnetwork to process the input features to obtain the first key K matrix, the first value V matrix and the first query Q (Query) matrix of each first decoding layer, and sequentially injecting the first key K matrix and the first value V matrix of each first decoding layer into the corresponding second decoding layer of the imitation U-Net subnetwork. The first key K matrix, the first value V matrix and the first query Q (Query) matrix of each first decoding layer are used to jointly determine the output features of the first decoding layer as the input features of the next decoding layer of the first decoding layer. The input features of the first decoding layer of the reference U-Net subnetwork are the first features, and the input features of each subsequent decoding layer are the output features of the previous decoding layer.

[0089] At the same time, by imitating multiple second encoding layers of the U-Net subnetwork, feature extraction is performed on its input (such as the first latent representation, the second latent representation and the aforementioned mask) in turn to obtain the second feature corresponding to the second sample image after occlusion processing; in each second decoding layer imitating the U-Net subnetwork, the input features are processed in turn to obtain the corresponding second key K matrix, second value V matrix and second query Q matrix; using the specified aggregation algorithm, the second key K matrix, second value V matrix and second query Q matrix corresponding to the second decoding layer, as well as the first key K matrix and first value V matrix corresponding to the second decoding layer, jointly determine the output features of the second decoding layer; then the output features of the second decoding layer are used as the input features of the next second decoding layer, until the second decoding layer is the last decoding layer, and the output image is obtained based on its output features.

[0090] For example, the first key K matrix and the first value V matrix of the first decoding layer of the reference U-Net sub-network are injected into the first second decoding layer of the imitation U-Net sub-network; the first key K matrix and the first value V matrix of the second decoding layer of the reference U-Net sub-network are injected into the second second decoding layer of the imitation U-Net sub-network, and so on.

[0091] In some examples, in order to maintain the shape of the sample body in the second sample image, a depth map of the unoccluded area in the second sample image can be obtained based on the depth estimation subnetwork in the style conversion network to be trained, and then a depth potential representation is obtained based on the depth map encoding, and the depth potential representation is also used as the input of the imitation U-Net subnetwork to obtain a better output image. Among them, the depth estimation subnetwork can be any network in the relevant technology that can determine the depth map corresponding to the image.

[0092] In step 24, based on the fourth background image and the output image, the parameters of the style transfer network to be trained are adjusted to obtain the style transfer network. In this step, a loss value can be constructed based on the difference between the content of the background area in the output image and the area corresponding to the background area in the fourth background image, and then the parameters of the style transfer network to be trained are adjusted with the goal of minimizing the loss value.

[0093] In some cases, the loss value can be constructed based on the pixel value difference loss, style loss and / or edge preservation loss (EdgePreservation Loss) between the content of the background area in the output image and the area corresponding to the background area in the fourth background image, so that the style transfer network can generate a generated image with better quality.

[0094] Furthermore, an image recognition network is trained based on the type of generated images (i.e., forged images) to obtain an image recognition network that can identify whether an input image is a real image.

[0095] Specifically, Figure 2 The flowchart of a sample generation method in one embodiment of the present specification is shown. The method is executed by an electronic device, and the electronic device can be implemented by any device, equipment, platform, equipment cluster, etc. with computing and processing capabilities. In the process of generating a card image, such as Figure 2 As shown, the method includes the following steps S210-S240:

[0096] In step S210, an image template is obtained, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type.

[0097] In some examples, the image template may be a card image template corresponding to the first card, and accordingly, the multiple layers are used to display user information, that is, the information of the specified type includes user information. In some other examples, the image template may also be a bill image template corresponding to certain bills, and accordingly, the multiple layers are used to display bill-related information, that is, the information of the specified type includes bill-related information. In some other examples, the image template may also be an image template corresponding to a certain type of book, an image template corresponding to a building, or an image template corresponding to a certain subject. Accordingly, the multiple layers included therein are used to display information of the specified type related to the subject type corresponding to the image template.

[0098] When the image template is a card image template corresponding to a first card, the first card may be any type of card. In some examples, the card mentioned in the embodiments of this specification may include but is not limited to at least one of the following: a bank card, various identity documents (such as an identity card, a passport, and a driver's license, etc.), etc.

[0099] In some possible examples, a preset storage area corresponding to the electronic device may pre-store an image template library, which may include pre-generated image templates corresponding to various types of cards and certificates. The user may select the image template of the first card and certificate he needs from the image template library based on his own needs. Accordingly, the electronic device obtains the card and certificate template corresponding to the first card and certificate selected by the user.

[0100] In some other possible examples, the embodiments of this specification further provide a process of obtaining an image template. For example, in step S210, steps 21-24 may be included:

[0101] In step 21, a first image is obtained, and each designated area of ​​the first image displays each designated type of information. Exemplarily, the first image may be a card image obtained by photographing a first card, and each designated area of ​​the first card displays each piece of information of a certain user. Another exemplary embodiment, the first image may be an image obtained by photographing another subject, and each designated area of ​​the other subject displays each designated type of information.

[0102] It is understandable that different types of entities have different designated areas and the designated types of information displayed therein. For example, a card generally includes information about the identity type of a person displayed in each designated area; information in a ticket generally includes information about the ticket displayed in each designated area; information in a book generally includes introductory information about the book displayed in each designated area, etc.

[0103] In the following, taking the first image as the card image obtained by photographing the first card (i.e., the subject is the card) as an example, the designated areas and the designated types of information displayed therein are introduced, wherein the designated areas in different types of cards may be different, and the types of user information (which may be referred to as the second information) displayed in each designated area may be different. For example: for a bank card, the designated area may include the area where the bank card number is located, and may also include the area where the name of the bank to which the bank card belongs is located. Accordingly, the second information may include the card number of the bank card (belonging to user text information), and may also include the name of the bank to which the bank card belongs (belonging to user text information). For another example, for an identity document, the designated area may include: the area where user text information such as the name, ID number, and related address of the person in the identity document is located, and may also include the area where the face image (belonging to image information) is located; accordingly, the second information may include the name, ID number, and related address of the person involved in the identity document for text information, as well as image information such as face image.

[0104] In some cases, the designated areas of different types of identity documents may be located at different positions on their surfaces. Accordingly, the designated areas of different types of identity documents may be located at different positions in images obtained by photographing the identity documents.

[0105] Next, in step 22, the regional images of the designated areas are detected and segmented from the first image to obtain the regional images and the background image after the regional images of the designated areas are segmented. In this step, the electronic device can determine the position information corresponding to the designated areas from the first image based on the designated target area detection algorithm, and then segment the regional images corresponding to the designated areas from the first image based on the position information corresponding to the designated areas, and use the first image after the regional images corresponding to the designated areas are segmented as the background image to obtain the regional images corresponding to the designated areas and the background image. At this time, the regional images corresponding to the designated areas are missing from the background image.

[0106] In the following, the first image is taken as a card image obtained by shooting the first card (i.e., the subject is the card) as an example for explanation. The regional image corresponding to each designated area may include various information in the user information in the first card, for example, including: designated character string and / or face image, wherein the designated character string includes but is not limited to: bank card number, name, relevant address, etc. The designated target area detection algorithm may be any type of detection algorithm in the related art that can detect foreground elements (the foreground elements include, for example, various types of information displayed in the designated area), and such detection algorithms may include but are not limited to: detection models based on deep learning, and algorithms based on traditional methods (for example: detection algorithms based on directional gradient histogram features, detection algorithms based on local binary pattern features, etc.).

[0107] In step 23, based on a preset background restoration algorithm, the background of the area corresponding to each regional image in the background image is restored to obtain a background layer of the first image. In this step, since the background image is segmented into various regional images, there is a problem of broken (or damaged) imaging content. In order to generate an image template with better quality, it is necessary to restore the broken imaging content in the background image. Accordingly, the electronic device can restore the background of the area corresponding to each regional image in the background image based on a preset background restoration algorithm and using the background content presented in other areas of the background image to obtain a background layer of the first image.

[0108] In some examples, the preset background restoration algorithm may be any algorithm that can repair broken content (damaged content) in an image, such as a Poisson cloning algorithm, or an image restoration algorithm based on a generative adversarial network.

[0109] Then, in step 24, based on the aforementioned background layer and each area image, each layer for displaying information of a specified type is determined.

[0110] In some examples, each area image may include not only the corresponding specified type of information (i.e., foreground elements) but also part of the background. In this case, the electronic device may refer to the position information of each area image in the first image, and determine the difference image corresponding to each area image based on each area image and the aforementioned background layer (background restoration has been performed). Each difference image includes the foreground elements in the corresponding area image, and each layer for displaying the specified type of information is determined based on each difference image to obtain an image template. In some cases, the image template may be stored in PSD (Photoshop Document) format.

[0111] In one case, a difference image may correspond to a layer for displaying a specified type of information, that is, there may be a one-to-one correspondence between the layer and the foreground element, each layer for displaying the specified type of information has the same size as the background layer, and the display area in each layer for displaying the specified type of information is determined based on the position information of the corresponding foreground element in the first image.

[0112] In another case, multiple difference images may correspond to one layer, the layer has the same size as the background layer, and the display area in the layer is determined based on the position information of each foreground element in the first image.

[0113] As another example, the electronic device may also be installed with image editing software. Based on the image editing function provided by the image editing software, the user may manually select the regional images corresponding to the designated regions from the first image, and trigger the layer creation function provided by the image editing software for each regional image to create a layer corresponding to each regional image. The layer corresponding to each regional image is used to display information of each designated type. Next, for the first image after separating the regional images, a layer is created, and the layer is processed (for example, background repair) to obtain a background layer to obtain an image template.

[0114] Thereafter, in step S220, first information for generating an image is obtained, where the first information includes information of a specified type corresponding to each layer.

[0115] In one implementation, a preset storage area of ​​the electronic device may pre-store information sets of each specified type corresponding to the subject corresponding to the image template, and accordingly, the electronic device may randomly select information of each specified type from the information sets of each specified type to form the first information, so as to generate an image containing new information based on the image template. In another implementation, the electronic device may use a feature program to randomly generate information of each specified type corresponding to the subject corresponding to the image template, i.e., the first information.

[0116] For example, the image template is a card image template corresponding to the first card. Assuming that the first card is a bank card, the types of information corresponding to the first card may include a bank card number, and the corresponding first information includes a randomly generated bank card number. Assuming that the first card is an identity document, the types of information corresponding to the first card may include, but are not limited to: user text information such as name, relevant address, and date of birth, and image information such as a user face image, and the corresponding first information may include a randomly generated name, relevant address, date of birth, and user face image.

[0117] In some possible examples, the first information may include: text information indicated by the image template and / or an image to be displayed.

[0118] Next, in step S230, a first generated image is generated based on the aforementioned image template and the first information.

[0119] The aforementioned image template includes layers for displaying each information in each specified type of information, and correspondingly, each layer has a corresponding relationship with each information in the first information. For example, the first information includes each user information related to the identity document, and a certain layer in the image template needs to display the name in the first information, and correspondingly, the layer has a corresponding relationship with the name in the first information. For another example, a certain layer in the image template needs to display the relevant address in the first information, and correspondingly, the layer has a corresponding relationship with the relevant address in the first information. In addition, each layer can indicate the display area of ​​each corresponding specified type of information.

[0120] The electronic device may generate a first generated image based on the correspondence between each layer in the image template and each information in the first information, the display area of ​​each information indicated by each layer, and each information in the first information and multiple layers in the image template. Figure 3A As shown, it is a schematic diagram of a frame of first generated image when the image template is a card image template, wherein the face image is not shown.

[0121] In some examples, each layer of the image template includes each specified type of information contained in the subject in the corresponding first image, that is, each layer of the image template may include its corresponding foreground element. The aforementioned process of generating the first generated image may include: the electronic device generates each layer displaying each information in the first information for each information in the first information, and based on the correspondence between each layer in the image template and each information in the first information, uses each layer displaying each information in the first information to replace each layer displaying each information in the image template (for example, uses the layer displaying the name in the first information to replace the layer displaying the name in the second information in the image template, and uses the layer displaying the date of birth in the first information to replace the layer displaying the date of birth in the second information in the image template) to generate the first generated image.

[0122] In some other examples, each layer in the image template may include a smart object (Smart Object) containing each foreground element (i.e., each information in the second information), wherein the smart object can be understood as a container, which may include objects such as images, graphics, and strings, and the position of the smart object in the layer is determined based on the position information of the foreground element corresponding to the layer in the first image. At this time, the aforementioned process of generating the first generated image may include: the electronic device replaces the elements contained in the smart object in each layer for each layer in the image template using the corresponding relationship between each layer in the image template and each information in the first information, using the corresponding information in the first information, so as to obtain each layer showing each information in the first information, and then generate the first generated image.

[0123] In some implementations, in order to better ensure the quality of the generated image, the display style of each information of the first information in the generated image can also be indicated based on the display style corresponding to each layer in the image template, so that each information in the generated image is more compatible with the background layer, and the generated image is more similar to the real image visually. Accordingly, in step S230, the following steps 31-32 may be included:

[0124] In step 31, the display style information corresponding to the image template is obtained, and the display style information is used to indicate the display style of each information in the first information in its corresponding layer. Exemplarily, the display style information may include, but is not limited to, style information such as font, font size, color, etc. corresponding to the text information, and may also include style information of the image to be displayed (such as information on the degree of edge blurring of the user's face image, etc.). Among them, for different text information, the corresponding style information may be the same or different. For example, the style information indicating the display style of the name in the first information in its corresponding layer may be font 1, font size 1, color 1; the style information indicating the display style of the relevant address in the first information in its corresponding layer may be font 2, font size 2, color 1, etc.

[0125] In step 32, a first generated image is generated based on the aforementioned image template, display style information and first information.

[0126] In some examples, the electronic device can generate layers displaying each information in the first information based on the correspondence between each layer in the aforementioned image template and each information in the first information, the display area of ​​each information corresponding to each layer indicated by the layer, the style information corresponding to each information in the first information in the display style information, and each information in the first information, wherein the information is displayed in the display area indicated by each layer in the display style corresponding to a certain information in the corresponding first information; and a first generated image is generated based on each layer displaying each information in the first information.

[0127] The generated first generated image takes the background layer in the aforementioned image template as the background, and displays each information in the display area indicated by the layer corresponding to each information in the first information in the display style corresponding to each information. Accordingly, the multiple layers of the image template also include the background layer.

[0128] After the first generated image is generated, in step S240, a plurality of second generated images are generated based on the first generated image through a style transfer network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image, and the plurality of second generated images are used as negative samples for training an image recognition network, and the image recognition network is used to identify whether an input image is a real image. The style transfer network may be a generation network in the aforementioned trained generation adversarial network, which has been trained based on the aforementioned training generation adversarial network process; or it may be the aforementioned trained MimicBrush-based style transfer network.

[0129] In some possible examples, the style transfer network is a generative network in the aforementioned generative adversarial network. Accordingly, in step S240, the following steps 41-43 may be included:

[0130] In step 41 , based on the first generated image, a first vector in the latent space is determined.

[0131] In this step, the electronic device can determine a first vector in the latent space based on the first generated image in a variety of ways, wherein the similarity between an image obtained by processing the first vector through a style transfer network and the first generated image exceeds a specified similarity threshold. The first vector includes the encoding information of the first generated image in the latent space.

[0132] For example, the electronic device can obtain a first vector in the latent space based on a pre-trained encoder and a first generated image. The pre-trained encoder can be trained based on each designated image A and its corresponding vector B in the latent space; each designated image A is an image generated by the style transfer network based on its corresponding vector B in the latent space. The subject in the designated image A can be the same type of subject as the subject corresponding to the image template. Exemplarily, the designated image is a card image, the subject is a card, the image template is a card image template, and the subject is a card. Another exemplary embodiment, the designated image is a bill image, the subject is a bill, the image template is a bill image template, and the subject is a bill.

[0133] For another example, the electronic device can determine its first vector in the latent space based on the first generated image by performing the following steps, wherein the steps include: sampling a vector in the latent space from the aforementioned predefined probability distribution (i.e., the probability distribution when training the aforementioned generative adversarial network) as an initial vector; processing the initial vector through the style transfer network to obtain an initial image, adjusting the initial vector with the goal of minimizing the difference between the initial image and the first generated image to obtain an adjusted initial vector; then processing the adjusted initial vector through the style transfer network again to obtain a new initial image; adjusting the adjusted initial vector with the goal of minimizing the difference between the new initial image and the first generated image, and so on, after multiple iterations, until the difference between the obtained new initial image and the first generated image is lower than a specified difference threshold (i.e., the similarity between the new initial image and the first generated image exceeds the specified similarity threshold), the new initial vector at this time is determined as the first vector. It can be understood that in the above process, the parameters of the style transfer network can be fixed.

[0134] In some possible examples, in order to generate more diverse generated images, background imaging content may be added to the first generated image, and then more diverse generated images may be obtained by changing the background. Accordingly, step 41 may include steps 411-412: In step 411, the first generated image is fused with the first background image to obtain a first fused image fused with the background indicated by the first background image. The first fused image includes the imaging content of the subject displaying the first information and the imaging content of the background in which the subject displaying the first information is located, such as Figure 3B As shown, the subject is a card, and the first information is the information of each user displayed on the card. In some possible examples, the first generated image can be fused with the first background image by specifying a fusion algorithm. The specified fusion algorithm may include, but is not limited to, any algorithm that can achieve image fusion in related technologies such as a Poisson fusion algorithm and an image fusion algorithm based on a deep learning network. The first background image may be an image uploaded by a user or randomly selected from a specified image library. The first background image may be an image obtained by shooting a desktop, a wall, or other types of backgrounds.

[0135] Next, in step 412, the first fused image is processed to obtain its first vector in the latent space. The implementation process of step 412 can refer to the implementation process of determining the first vector in the latent space based on the first generated image, which will not be described in detail here.

[0136] After obtaining the aforementioned first vector, in step 42, based on the correspondence between each position and each style in the vector of the latent space recorded in advance, the parameter value of the specified position in the first vector is adjusted to obtain several second vectors after adjusting the styles corresponding to the specified positions. The styles corresponding to the specified positions are used to indicate the generation of the environmental background.

[0137] After obtaining the trained style transfer network, the electronic device can explore the correspondence between each position and each style in the vector of the latent space by randomly or systematically adjusting the encoded information (i.e., the vector used to generate the card image) in the latent space, and record the correspondence. After obtaining the first vector based on the first generated image, the parameter value of the specified position in the first vector can be adjusted based on the correspondence between each position and each style in the vector of the latent space to obtain a number of second vectors after adjusting the style corresponding to the specified position, and the style corresponding to the specified position is used to indicate the generation of the environmental background.

[0138] In some examples, the environmental background may include: a lighting display pattern in the image. In some other examples, the aforementioned environmental background may also include the background in the aforementioned first fused image.

[0139] After obtaining the plurality of second vectors, in step 43, a plurality of second generated images are generated based on the plurality of second vectors through a style conversion network. The plurality of second generated images have different environmental backgrounds relative to the first generated image. In this step, the electronic device inputs each second vector into the style conversion network, processes the input second vector through the style conversion network, and obtains a second generated image corresponding to the second vector. With respect to the first vector, the parameter values ​​of the positions for indicating the generation of the environmental background of each second vector are different, and the parameter values ​​of other positions are the same. Accordingly, the plurality of second generated images have different environmental backgrounds relative to the first generated image, and the first information is displayed in each of them.

[0140] Exemplarily, if the style corresponding to the aforementioned designated position is a lighting condition, the plurality of second generated images may have different lighting display styles relative to the first generated image. Also exemplarily, if the style corresponding to the aforementioned designated position is a background condition, the plurality of second generated images may have different backgrounds relative to the aforementioned first fused image.

[0141] In some other possible examples, the style transfer network is the aforementioned Mimic Brush-based grid transfer network. Accordingly, in step S240, the following steps 51-53 may be included:

[0142] In step 51, the first generated image is fused with the second background image to obtain a second fused image fused with the background indicated by the second background image. The second background image can be an image uploaded by a user or randomly selected from a specified image library. The first background image can be an image obtained by shooting a desktop, a wall, or other types of backgrounds. The implementation principle of fusing the first generated image with the second background image is similar to the implementation principle of fusing the first generated image with the first background image. The implementation process can refer to the implementation process of fusing the first generated image with the first background image, which will not be described in detail here.

[0143] In step 52, the area where the background is located in the second fused image is blocked to obtain the second fused image after the blocking process. In this step, a mask (or mask) corresponding to the area where the background is located in the second fused image can be created to block the area where the background is located in the second fused image to obtain the second fused image after the blocking process, wherein the second fused image after the blocking process includes the second fused image and its corresponding mask.

[0144] Then, in step 53, a style transfer network is used to obtain a plurality of second generated images based on the second fused image after occlusion processing and a plurality of third background images, wherein the background in each second generated image is generated based on the content in each third background image.

[0145] In this step, for each third background image, the third background image is processed by the first encoding subnetwork of the style transfer network to obtain a potential representation corresponding to the third background image, and the potential representation corresponding to the third background image is input into the reference U-Net subnetwork of the style transfer network; the second fused image after occlusion processing (i.e., the second fused image and its corresponding mask) is processed by the second encoding subnetwork of the style transfer network to obtain a potential representation corresponding to the second fused image and a potential representation of an unoccluded area in the second fused image, and both of them are input into the imitation U-Net subnetwork of the style transfer network, so as to jointly process the potential representation corresponding to the third background image, the potential representation corresponding to the second fused image, and the potential representation of the unoccluded area in the second fused image through the reference U-Net subnetwork and the imitation U-Net subnetwork to obtain a second generated image corresponding to the third background image, wherein the background in the second generated image (i.e., the content of the area occluded by the mask corresponding to the second fused image) is generated based on the third background image, and so on, to obtain a plurality of second generated images.

[0146] Subsequently, the plurality of second generated images can be used as negative samples for training the image recognition network. In some other examples, the aforementioned first generated images can also be used as negative samples for training the image recognition network.

[0147] In the above process, a first generated image can be generated through an image template and the first information, and then the first generated image can be processed through a style transfer network to transform the environmental background of the first generated image, thereby generating a number of second generated images with different environmental backgrounds from the first generated image, thereby achieving the generation of more diverse generated images based on artificial intelligence technology, and then using a number of second generated images (i.e., forged images generated based on artificial intelligence technology) as negative samples for training an image recognition network to train the image recognition network, thereby helping to improve the image recognition network's ability to recognize generated images (i.e., forged images generated based on artificial intelligence technology), thereby protecting the network environment, helping users distinguish the authenticity of images in the network environment, and improving user experience.

[0148] Corresponding to the above method embodiment, the present specification embodiment also provides a method for training an image recognition network, which is performed by an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. In the process of training the image recognition network, if Figure 4 As shown, the method includes the following steps S410-S450:

[0149] In step S410, an image template is obtained, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type.

[0150] In step S420, first information for generating an image is obtained, where the first information includes information of a specified type corresponding to each layer.

[0151] In step S430, a first generated image is generated based on the aforementioned image template and the first information.

[0152] In step S440, a style transfer network is used to generate a plurality of second generated images based on the first generated image, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image.

[0153] Among them, the implementation principle of steps S410-S440 is similar to the implementation principle of the aforementioned steps S210-S240. The implementation process can refer to the aforementioned steps S210-S240 and will not be repeated here.

[0154] After obtaining a plurality of second generated images, in step S450, an image recognition network is trained based on the plurality of second generated images, wherein the image recognition network is used to identify whether an input image is a real image.

[0155] In this step, the image recognition network is trained based on the aforementioned several second generated images and their corresponding label data indicating that the second generated images are forged images. The image recognition network can be a neural network based on deep learning. In some other possible examples, the electronic device can train the image recognition network based on the aforementioned several second generated images and their corresponding label data indicating that the second generated images are forged images, as well as several collected real images and their corresponding label data indicating that the real images are real images, so as to train the image recognition network to reach the convergence condition. Among them, the convergence condition may include but is not limited to: the training time reaches a preset time, the number of times the parameters of the image recognition network are adjusted reaches a specified number of times, or the loss calculated based on the label data and the predicted data output by the image recognition network for the input image (including the several second generated images and the several collected real images) is lower than a preset threshold.

[0156] After the image recognition network is trained to reach a preset convergence condition, the image recognition network can be used to perform authenticity identification on the input image.

[0157] Corresponding to the above method embodiment, the present specification embodiment also provides a method for training a card classification network, which is executed by an electronic device, and the electronic device can be implemented by any device, equipment, platform, device cluster, etc. with computing and processing capabilities. In the process of training the card classification network, if Figure 5 As shown, the method includes the following steps S510-S550:

[0158] In step S510, a card template corresponding to the first card is obtained, wherein the card template includes multiple layers, and the multiple layers are used to display user information.

[0159] In step S520, first user information for generating a card image is obtained.

[0160] In step S530, a first card image is generated based on the aforementioned card template and the first user information.

[0161] In step S540, a plurality of second card images are generated based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image.

[0162] Among them, the implementation principle of steps S510-S540 is similar to the implementation principle of the aforementioned steps S210-S240. The implementation process can refer to the aforementioned steps S210-S240 and will not be repeated here.

[0163] After obtaining the plurality of second card images, in step S550, a card classification network is trained based on the plurality of second card images, wherein the card classification network is used to identify whether the input card image is a real card image. In this step, the card classification network is trained based on the plurality of second generated images and their corresponding second label data indicating that the second generated image is a forged card image. The card classification network may be a neural network based on deep learning.

[0164] Specifically, in some possible examples, step S550 may include the following steps 61-62:

[0165] In step 61, the second card image is processed through the card classification network to obtain classification data for determining whether the second card image is a real card image. In this step, the electronic device inputs the second card image into the card classification network, processes the second card image through the card classification network, and outputs classification data for determining whether the second card image is a real card image. The classification data may be a probability value for determining that the second card image is a real card image. Exemplarily, the larger the probability value, the greater the possibility that the second card image is a real card image.

[0166] Next, in step 62, based on the second label data and the classification data corresponding to the second card image, the parameters of the card classification network are adjusted. In this step, the electronic device constructs a classification loss based on the difference between the second label data and the classification data corresponding to the second card image, and adjusts the parameters of the card classification network with the goal of minimizing the classification loss.

[0167] It can be understood that the electronic device can also take pictures of real card certificates to collect images of real card certificates (i.e., real card certificate images), and obtain second label data corresponding to each real card certificate image indicating that it is a real card certificate image, and combine the real card certificate image and its corresponding second label data to train the card classification network to improve the stability and accuracy of the classification results of the card classification network.

[0168] It is understandable that the parameters of the card classification network can be adjusted through multiple iterative training processes until the card classification network reaches a preset convergence condition, thereby obtaining a trained card classification network. The preset convergence condition may include but is not limited to: the number of iterations exceeds a preset number threshold; the training time exceeds a specified time; the classification loss is lower than a preset loss threshold, etc.

[0169] The card classification network can then be used to provide a classification and identification service for determining whether the card image is a genuine card image.

[0170] In this embodiment, a first card image can be generated through a card template and first user information, and then the first card image is processed through a style conversion network to transform the environmental background of the first card image, thereby generating a plurality of second card images with environmental backgrounds different from those of the first card image, thereby achieving generation of more diverse card images based on artificial intelligence technology, and then using the plurality of second card images (i.e., card images generated based on artificial intelligence technology) as negative samples to train a card classification network, so that the card classification network has the ability to identify whether an input card image is a real card image, that is, a card classification network that can be trained to identify forged card images is achieved.

[0171] Moreover, the second card image generated based on the style transfer network has the characteristics of an image generated based on artificial intelligence technology. The card classification network trained based on such a second card image can better realize the recognition of card images (forged card images) generated based on artificial intelligence technology, better help the eKYC (electronic know your customer) system to avoid attacks on card images generated based on artificial intelligence technology, intercept such attacks, and better improve the security performance of the eKYC system.

[0172] The foregoing describes certain embodiments of the present specification, and other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in an order different from that in the embodiments, and the desired results may still be achieved. In addition, the processes depicted in the accompanying drawings do not necessarily have to be performed in the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0173] Corresponding to the above method embodiment, this specification embodiment provides a sample generation device 600, whose schematic block diagram is as follows: Figure 6 As shown, including:

[0174] A first acquisition module 610 is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type;

[0175] A second acquisition module 620 is configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0176] A first generating module 630, configured to generate a first generated image based on the image template and the first information;

[0177] The second generation module 640 is configured to generate a number of second generated images based on the first generated image through a style transfer network, wherein the number of second generated images have different environmental backgrounds relative to the first generated image, and the number of second generated images serve as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image.

[0178] In some possible examples, the first generating module 630 is specifically configured to obtain display style information corresponding to the image template, where the display style information is used to indicate a display style of each information in the first information in its corresponding layer;

[0179] The first generated image is generated based on the image template, the display style information and the first information.

[0180] In some possible examples, the first generated image takes the background layer in the image template as the background, and each information in the first information is displayed in a display area indicated by the layer corresponding to each information in a display style corresponding to the information.

[0181] In some possible examples, the first information includes: text information indicated by the image template and / or an image to be displayed.

[0182] In some possible examples, the first acquisition module 610 is specifically configured to acquire a first image, each designated area of ​​the first image displays each information of each designated type;

[0183] Detecting and segmenting the regional images of the designated regions from the first image to obtain the regional images and the background image after segmenting the regional images of the designated regions;

[0184] Based on a preset background restoration algorithm, performing background restoration on the area corresponding to each area image in the background image to obtain a background layer of the first image;

[0185] Based on the background layer and each area image, each layer for displaying information of a specified type is determined.

[0186] In some possible examples, the second generating module 640 includes:

[0187] a determining unit (not shown in the figure), configured to determine a first vector of the first generated image in a latent space based on the first generated image;

[0188] an adjusting unit (not shown in the figure), configured to adjust the parameter value of the specified position in the first vector based on the correspondence between each position and each style in the vector of the latent space recorded in advance, to obtain a plurality of second vectors after adjusting the style corresponding to the specified position, wherein the style corresponding to the specified position is used to indicate the generation of the environmental background;

[0189] A generating unit (not shown in the figure) is configured to generate a plurality of second generated images based on a plurality of second vectors through the style conversion network.

[0190] In some possible examples, the determining unit is specifically configured to fuse the first generated image with a first background image to obtain a first fused image fused with the background indicated by the first background image;

[0191] The first fused image fused with the background is processed to obtain a first vector in a latent space.

[0192] In some possible examples, the environmental background includes: a lighting display pattern in an image and / or a background in the first fused image fused with the background.

[0193] Some possible examples include:

[0194] A third acquisition module (not shown in the figure) is configured to acquire a generative adversarial network, wherein the generative adversarial network includes a discriminant network and a style transfer network to be trained, and the discriminant network has been trained based on each first sample image and its corresponding first label data indicating whether it is a real image;

[0195] a fourth acquisition module (not shown in the figure), configured to acquire a second vector in the latent space;

[0196] A first obtaining module (not shown in the figure) is configured to obtain a current generated image based on the second vector through a style conversion network to be trained;

[0197] A second obtaining module (not shown in the figure) is configured to obtain a discrimination result of whether the current generated image is a real image based on the current generated image through the discrimination network;

[0198] The first adjustment module (not shown in the figure) is configured to adjust the parameters of the style conversion network to be trained based on the discrimination result to obtain the style conversion network.

[0199] In some possible examples, the style transfer network is a MimicBrush-based style transfer network;

[0200] The second generating module 640 is specifically configured to fuse the first generated image with the second background image to obtain a second fused image fused with the background indicated by the second background image;

[0201] Blocking the background area in the second fused image to obtain a second fused image after the blocking process;

[0202] Through the style transfer network, based on the second fused image after the occlusion processing and a plurality of third background images, a plurality of second generated images are obtained, wherein each second generated image has the background in each third background image.

[0203] Some possible examples include:

[0204] A fifth acquisition module (not shown in the figure) is configured to acquire a second sample image and a fourth background image corresponding thereto, wherein the second sample image is an image obtained by photographing the sample body and the background in which it is located;

[0205] an occlusion module (not shown in the figure), configured to perform an occlusion process on the second sample image to occlude a background area in the second sample image;

[0206] an input module (not shown in the figure), configured to input the fourth background image and the second sample image after occlusion processing into a style conversion network to be trained, so as to replace the content of the background area with the content in the fourth background image to obtain an output image;

[0207] The second adjustment module (not shown in the figure) is configured to adjust the parameters of the style conversion network to be trained based on the fourth background image and the output image to obtain the style conversion network.

[0208] Corresponding to the above method embodiment, the present specification embodiment provides a device 700 for training an image recognition network, and its schematic block diagram is as follows: Figure 7 As shown, including:

[0209] The third acquisition module 710 is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type;

[0210] A fourth acquisition module 720 is configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer;

[0211] A third generating module 730 is configured to generate a first generated image based on the image template and the first information;

[0212] A fourth generating module 740 is configured to generate a plurality of second generated images based on the first generated image through a style conversion network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image;

[0213] The first training module 750 is configured to train an image recognition network based on a plurality of second generated images, wherein the image recognition network is used to identify whether an input image is a real image.

[0214] Corresponding to the above method embodiment, the present specification embodiment provides a device 800 for training a card classification network, and its schematic block diagram is as follows: Figure 8 As shown, including:

[0215] A fifth acquisition module 810 is configured to acquire a card template corresponding to the first card, wherein the card template includes a plurality of layers, and the plurality of layers are used to display user information;

[0216] A sixth acquisition module 820 is configured to acquire first user information for generating a card image;

[0217] A fifth generating module 830 is configured to generate a first card image based on the card template and the first user information;

[0218] A sixth generation module 840 is configured to generate a plurality of second card images based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image;

[0219] The second training module 850 is configured to train a card classification network based on the plurality of second card images, wherein the card classification network is used to identify whether an input card image is a real card image.

[0220] In some possible examples, the first training module 650 is specifically configured to process the second card image through the card classification network to obtain classification data for determining whether the second card image is a real card image;

[0221] Based on the second label data corresponding to the second card image and the classification data, the parameters of the card classification network are adjusted.

[0222] The above device embodiments correspond to the method embodiments. For specific descriptions, please refer to the description of the method embodiments, which will not be repeated here. The device embodiments are obtained based on the corresponding method embodiments and have the same technical effects as the corresponding method embodiments. For specific descriptions, please refer to the corresponding method embodiments.

[0223] The embodiments of this specification also provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed in a computer, the computer is caused to execute the method of sample generation, the method of training an image recognition network, or the method of training a card classification network provided in this specification.

[0224] The embodiments of the present specification also provide a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method of sample generation, the method of training image recognition network or the method of training card classification network provided in the present specification is implemented.

[0225] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the storage medium and computing device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0226] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present invention may be implemented using hardware, software, firmware, or any combination thereof. When implemented using software, these functions may be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium.

[0227] The specific implementation methods described above further describe the purpose, technical solutions and beneficial effects of the embodiments of the present invention in detail. It should be understood that the above description is only a specific implementation method of the embodiments of the present invention and is not intended to limit the scope of protection of the present invention. Any modification, equivalent replacement, improvement, etc. made on the basis of the technical solution of the present invention shall be included in the scope of protection of the present invention.

Claims

1. A sample generation method, comprising: Acquire an image template, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type; Acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer; Based on the image template and the first information, generate a first generated image; Based on the first generated image, a number of second generated images are generated through a style transfer network, wherein the number of second generated images have different environmental backgrounds relative to the first generated image, and the number of second generated images are used as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image.

2. The method of claim 1, wherein: The step of generating a first generated image based on the image template and the first information includes: Acquire display style information corresponding to the image template, where the display style information is used to indicate a display style of each information in the first information in its corresponding layer; The first generated image is generated based on the image template, the display style information and the first information.

3. The method of claim 2, wherein: The first generated image takes the background layer in the image template as the background, and displays each information in the first information in a display area indicated by the layer corresponding to each information in a display style corresponding to the information.

4. The method of claim 1, wherein: The first information includes: text information indicated by the image template and / or the image to be displayed.

5. The method of claim 1, wherein: The step of obtaining an image template comprises: Acquire a first image, wherein each designated area of ​​the first image displays each designated type of information; Detecting and segmenting the regional images of the designated regions from the first image to obtain the regional images and the background image after segmenting the regional images of the designated regions; Based on a preset background restoration algorithm, performing background restoration on the area corresponding to each area image in the background image to obtain a background layer of the first image; Based on the background layer and each area image, each layer for displaying information of a specified type is determined.

6. The method of claim 1, wherein: The step of generating a plurality of second generated images based on the first generated image through a style conversion network includes: Based on the first generated image, determine a first vector in a latent space; Based on the pre-recorded correspondence between each position and each style in the vector of the latent space, adjusting the parameter value of the specified position in the first vector to obtain a plurality of second vectors after adjusting the styles corresponding to the specified positions, wherein the styles corresponding to the specified positions are used to indicate the generation of the environment background; A plurality of second generated images are generated based on the plurality of second vectors through the style transfer network.

7. The method of claim 6, wherein: The step of determining a first vector in a latent space based on the first generated image includes: Fusing the first generated image with a first background image to obtain a first fused image fused with the background indicated by the first background image; The first fused image is processed to obtain a first vector thereof in a latent space.

8. The method of claim 7, wherein: The environmental background includes: a lighting display pattern in an image and / or a background in the first fused image.

9. The method according to any one of claims 1 to 8, further comprising: Obtaining a generative adversarial network, wherein the generative adversarial network includes a discriminant network and a style transfer network to be trained, wherein the discriminant network has been trained based on each first sample image and its corresponding first label data indicating whether it is a real image; Get the third vector in the latent space; Obtaining a current generated image based on the third vector through the style conversion network to be trained; Obtaining, through the discriminant network, a discriminant result of discriminating whether the currently generated image is a real image based on the currently generated image; Based on the discrimination result, the parameters of the style conversion network to be trained are adjusted to obtain the style conversion network.

10. The method according to any one of claims 1 to 8, wherein: The style transfer network is a MimicBrush-based style transfer network; The step of generating a plurality of second generated images based on the first generated image through a style conversion network includes: Fusing the first generated image with a second background image to obtain a second fused image fused with the background indicated by the second background image; Blocking the background area in the second fused image to obtain a second fused image after the blocking process; Through the style transfer network, based on the second fused image after the occlusion processing and a plurality of third background images, a plurality of second generated images are obtained, wherein each second generated image has the background in each third background image.

11. The method of claim 10, further comprising: Acquire a second sample image and a fourth background image corresponding thereto, wherein the second sample image is an image obtained by photographing the sample subject and the background in which it is located; performing an occlusion process on the second sample image to occlude a background area in the second sample image; Inputting the fourth background image and the second sample image after occlusion processing into a style conversion network to be trained, so as to replace the content of the background area with the content in the fourth background image, and obtain an output image; Based on the fourth background image and the output image, parameters of the style conversion network to be trained are adjusted to obtain the style conversion network.

12. A method for training an image recognition network, comprising: Acquire an image template, wherein the image template includes multiple layers, and the multiple layers are used to display information of a specified type; Acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer; Based on the image template and the first information, generate a first generated image; generating, based on the first generated image, a plurality of second generated images through a style conversion network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image; Based on the plurality of second generated images, an image recognition network is trained, wherein the image recognition network is used to identify whether an input image is a real image.

13. A method for training a card classification network, comprising: Obtaining a card template corresponding to the first card, wherein the card template includes multiple layers, and the multiple layers are used to display user information; Acquire first user information for generating a card image; Generate a first card image based on the card template and the first user information; generating a plurality of second card images based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image; Based on the plurality of second card images, a card classification network is trained, wherein the card classification network is used to identify whether an input card image is a real card image.

14. The method of claim 13, wherein: The step of training a card classification network based on the plurality of second card images comprises: Processing the second card image through the card classification network to obtain classification data for determining whether the second card image is a real card image; Based on the second label data corresponding to the second card image and the classification data, the parameters of the card classification network are adjusted.

15. A sample generating device, comprising: A first acquisition module is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type; A second acquisition module is configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer; A first generating module, configured to generate a first generated image based on the image template and the first information; The second generation module is configured to generate a plurality of second generated images based on the first generated image through a style transfer network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image, and the plurality of second generated images serve as negative samples for training an image recognition network, and the image recognition network is used to identify whether the input image is a real image.

16. A device for training an image recognition network, comprising: A third acquisition module is configured to acquire an image template, wherein the image template includes a plurality of layers, and the plurality of layers are used to display information of a specified type; a fourth acquisition module, configured to acquire first information for generating an image, wherein the first information includes information of a specified type corresponding to each layer; A third generating module, configured to generate a first generated image based on the image template and the first information; a fourth generating module, configured to generate a plurality of second generated images based on the first generated image through a style conversion network, wherein the plurality of second generated images have different environmental backgrounds relative to the first generated image; The first training module is configured to train an image recognition network based on a plurality of second generated images, wherein the image recognition network is used to identify whether an input image is a real image.

17. A device for training a card classification network, comprising: a fifth acquisition module, configured to acquire a card template corresponding to the first card, wherein the card template includes a plurality of layers, and the plurality of layers are used to display user information; A sixth acquisition module, configured to acquire first user information for generating a card image; a fifth generating module, configured to generate a first card image based on the card template and the first user information; a sixth generation module, configured to generate a plurality of second card images based on the first card image through a style conversion network, wherein the plurality of second card images have different environmental backgrounds relative to the first card image; The second training module is configured to train a card classification network based on the plurality of second card images, wherein the card classification network is used to identify whether an input card image is a real card image.

18. A computing device comprising a memory and a processor, wherein: The memory stores executable codes, and when the processor executes the executable codes, the method according to any one of claims 1 to 14 is implemented.

Images