Image processing and model training method and apparatus, device, storage medium, and program

Abstract

The present application provides an image processing and model training method and apparatus, a device, a storage medium, and a program, which are applied to fields such as AI drawing and model training. The method comprises: obtaining a first biometric feature image and a prompt word (S201), the prompt word being used for indicating a preset living body attack type; extracting first image feature information of the first biometric feature image and text feature information of the prompt word (S202); determining second image feature information corresponding to the preset living body attack type (S203); and inputting the text feature information, the first image feature information, and the second image feature information into an image generation model and, on the basis of the first image feature information and by means of the image generation model, generating a first living body attack image conforming to the text feature information and the second image feature information (S204). That is, according to the present application, the first biometric feature image is processed by using the second image feature information and the prompt word as control conditions, thereby improving generation quality of the first living body attack image.

Description

Image processing and model training methods, apparatus, equipment, storage media and programs Technical Field

[0001] This application claims priority to Chinese Patent Application No. 2024110417895, filed on July 31, 2024, entitled “Image Processing and Model Training Method, Apparatus, Device and Storage Medium”, the entire contents of which are incorporated herein by reference.

[0002] Technical Field

[0003] This application relates to the field of computer technology, and in particular to an image processing and model training method, apparatus, device, storage medium, and program. Background Technology

[0004] With the rapid development of Artificial Intelligence (AI) technology, AI drawing applications have emerged. AI drawing includes two application modes: text-to-image and image-to-image. The application scenarios of AI drawing are very broad. For example, during model training, training images can be generated using AI drawing. For instance, in the training of a liveness detection model, the types of liveness attacks are complex and numerous, and relying solely on manual data collection and annotation cannot meet actual business needs. In this case, AI drawing can be used to generate training images corresponding to specific liveness attack types.

[0005] However, currently, when generating another image based on one image, the quality of the generated image is not ideal and cannot meet the expected requirements. Summary of the Invention

[0006] This application provides an image processing and model training method, apparatus, device, storage medium, and program that can improve the quality of generated images.

[0007] In a first aspect, this application provides an image processing method, comprising:

[0008] Acquire a first biometric image and a prompt word, the prompt word indicating a preset live attack type;

[0009] Extract the first image feature information from the first biometric image and the text feature information from the prompt word;

[0010] Determine the second image feature information corresponding to the preset liveness attack type;

[0011] The text feature information, the first image feature information, and the second image feature information are input into the image generation model. The image generation model is set to generate a first liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0012] Secondly, this application provides a model training method, including:

[0013] Acquire training images and prompt words, wherein the prompt words indicate a preset live attack type, and the training images are biometric images or live attack images;

[0014] Extract the first image feature information of the training image and the text feature information of the prompt word;

[0015] Determine the second image feature information corresponding to the preset liveness attack type;

[0016] The text feature information of the prompt word, the first image feature information, and the second image feature information are input into the image generation model. The image generation model is configured to generate a third liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0017] Based on the third liveness attack image and the feature information of the second image, the loss of the image generation model is determined, and the image generation model is trained based on the loss.

[0018] Thirdly, this application provides an image processing apparatus, comprising:

[0019] The acquisition unit is used to acquire a first biometric image and a prompt word, wherein the prompt word indicates a preset live attack type;

[0020] The feature extraction unit is used to extract the first image feature information of the first biometric image and the text feature information of the prompt word;

[0021] The determining unit is used to determine the second image feature information corresponding to the preset liveness attack type;

[0022] The processing unit is configured to input the text feature information, the first image feature information, and the second image feature information into an image generation model, wherein the image generation model is configured to generate a first liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0023] In some embodiments, the determining unit is specifically used to determine N candidate image feature information corresponding to the preset liveness attack type, wherein the N candidate image feature information are image feature information of N kinds of liveness attacks belonging to the preset liveness attack type, and N is a positive integer; and select one candidate parameter feature information from the N candidate image feature information as the second image feature information.

[0024] In some embodiments, the N candidate image feature information or the live attack feature information of the second live attack image is image texture feature information.

[0025] In some embodiments, the determining unit is specifically used to obtain a second live attack image based on the preset live attack type; and to extract the live attack feature information of the second live attack image as the second image feature information.

[0026] In some embodiments, the image generation model includes a denoising network and a processing unit, specifically configured to: add noise to the first image feature information of the first biometric image to obtain denoised feature information of the first biometric image; denoise the i-th denoised feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information through the denoising network to obtain the i-th noise prediction value, where i is a positive integer less than or equal to N; if i is 1, the i-th denoised feature information is the denoised feature information of the first biometric image; obtain the (i+1)-th denoised feature information of the first biometric image based on the i-th noise prediction value and the i-th denoised feature information of the first biometric image; use the (i+1)-th denoised feature information as the new i-th denoised feature information, and repeat the above steps to obtain the N-th denoised feature information of the first biometric image; determine the N-th denoised feature information as the denoised image feature information; and decode the denoised image feature information to obtain the first live attack image.

[0027] In some embodiments, the processing unit is specifically configured to use the i-th denoising feature information of the first biometric image as the input feature information of the denoising network; to fuse the feature information output by the first part of the network layer of the denoising network with at least the text feature information of the prompt word; and to fuse the feature information output by the second part of the network layer of the denoising network with at least the second image feature information to obtain the i-th noise prediction value.

[0028] In some embodiments, the processing unit is specifically configured to, for the kth network layer in the denoising network, fuse the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network, where k is a positive integer; and obtain the i-th noise prediction value based on the input feature information of the (k+1)th network layer.

[0029] In some embodiments, the processing unit is specifically used to take the text feature information of the prompt word as the key value of the attention module, take the second image feature information as the representative value of the attention module, take the output feature information of the kth network layer as the query value of the attention module, and input them into the attention module for fusion processing to obtain the input feature information of the (k+1)th network layer.

[0030] In some embodiments, the feature extraction unit is specifically used to preprocess the first biometric image to obtain a preprocessed first biometric image; and to extract image features from the preprocessed first biometric image to obtain first image feature information of the first biometric image.

[0031] In some embodiments, the feature extraction unit is specifically used to perform image cutout processing on the first biometric image to cut out the object region in the first biometric image; adjust the size of the object region to a first size; and crop the object region of the first size to a second size to obtain the preprocessed first biometric image.

[0032] Fourthly, this application provides a model training apparatus, comprising:

[0033] An acquisition unit is used to acquire training images and prompt words, wherein the prompt words indicate a preset live attack type, and the training images are biometric images or live attack images;

[0034] The feature extraction unit is used to extract the first image feature information of the training image and the text feature information of the prompt word;

[0035] The determining unit is used to determine the second image feature information corresponding to the preset liveness attack type;

[0036] The processing unit is configured to input the text feature information of the prompt word, the first image feature information, and the second image feature information into an image generation model, wherein the image generation model is configured to generate a third liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0037] The training unit is used to determine the loss of the image generation model based on the third liveness attack image and the feature information of the second image, and to train the image generation model based on the loss.

[0038] In some embodiments, the determining unit is specifically used to determine N candidate image feature information corresponding to the preset liveness attack type, wherein the N candidate image feature information are image feature information of N kinds of liveness attacks belonging to the preset liveness attack type, and N is a positive integer; and select one candidate image feature information from the N candidate image feature information as the second image feature information.

[0039] In some embodiments, the determining unit is specifically configured to extract the live attack feature information of the training image as the second image feature information if the training image is a live attack image.

[0040] In some embodiments, the determining unit is specifically used to obtain a fourth live attack image based on the preset live attack type, and extract the live attack feature information of the fourth live attack image as the second image feature information.

[0041] In some embodiments, the training unit is specifically used to extract live attack features from the third live attack image to obtain live attack feature information of the third live attack image; and to determine the loss of the image generation model based on the second image feature information and the live attack feature information of the third live attack image.

[0042] In some embodiments, if the second image feature information is the texture feature information of the training image, then the training unit is specifically used to extract texture features from the third live attack image to obtain the texture feature information of the third live attack image; and to determine the loss of the image generation model based on the texture feature information of the training image and the texture feature information of the third live attack image.

[0043] In some embodiments, the training unit specifically determines the difference between the texture feature information of the training image and the texture feature information of the third liveness attack image; and determines the loss of the image generation model based on the difference.

[0044] In some embodiments, the feature extraction unit is specifically used to preprocess the training image to obtain a preprocessed training image; and to extract image features from the preprocessed training image to obtain the first image feature information of the training image.

[0045] In some embodiments, the feature extraction unit is specifically used to perform image matting on the training image to extract object regions in the training image; adjust the size of the object regions to a first size; and randomly crop the object regions of the first size to a second size to obtain the preprocessed training image.

[0046] Fifthly, this application provides an electronic device including a processor and a memory. The memory is used to store a computer program, and the processor is used to invoke and run the computer program stored in the memory to perform the methods described in the first or second aspect above.

[0047] In a sixth aspect, a chip is provided for implementing the methods of various implementations of the first aspect described above. Specifically, the chip includes a processor for retrieving and running a computer program from a memory, causing a device equipped with the chip to perform the methods of the first or second aspect described above.

[0048] In a seventh aspect, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the methods described in the first or second aspect.

[0049] Eighthly, a computer program product is provided, including computer program instructions that cause a computer to perform the methods described in the first or second aspect.

[0050] Ninthly, a computer program is provided that, when run on a computer, causes the computer to perform the methods of the first or second aspect described above.

[0051] In summary, this application acquires a first biometric image and a prompt word, the prompt word indicating a preset live attack type; it extracts image features from the first biometric image to obtain first image feature information, and extracts text features from the prompt word to obtain text feature information, and determines second image feature information corresponding to the preset live attack type. Then, it inputs the text feature information, the first image feature information, and the second image feature information into an image generation model, which generates a first live attack image that conforms to the text feature information and the second image feature information based on the first image feature information. Therefore, this application embodiment, when generating a first live attack image based on a first biometric image and a prompt word, uses the second image feature information and the prompt word as control conditions to process the first biometric image and generate the first live attack image. This ensures that the generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word but also includes the image features corresponding to the preset live attack type, thereby improving the generation quality of the first live attack image. If the above image processing method is applied to generate training data for a liveness detection model, realistic and high-quality liveness attack samples can be generated. When the liveness detection model is trained using these realistic and high-quality liveness attack samples, the accuracy of the liveness detection model in recognizing realistic liveness attack data can be improved, thereby enhancing the detection performance of the liveness detection model. Attached Figure Description

[0052] Figure 1 is a schematic diagram of the implementation environment of an image processing method provided in an embodiment of this application;

[0053] Figure 2 is a schematic flowchart of a model training method provided in an embodiment of this application;

[0054] Figure 3 is a schematic diagram of a graphic representation;

[0055] Figure 4 is a schematic diagram of image feature information extraction;

[0056] Figure 5 is a schematic diagram of text feature information extraction;

[0057] Figure 6 is a schematic diagram of the image processing process involved in an embodiment of this application;

[0058] Figure 7 is a schematic diagram of an image generation model;

[0059] Figure 8 is a schematic diagram of denoising using a denoising network;

[0060] Figure 9 is a schematic diagram of a denoising network structure;

[0061] Figure 10 is a schematic diagram of data processing between different network layers in the denoising network;

[0062] Figures 11, 12, and 13 are schematic diagrams illustrating the fusion of the output feature information of the denoising network layer with the text feature information of the prompt words and the feature information of the second image.

[0063] Figures 14A, 14B, and 14C are schematic diagrams illustrating the fusion of the k-th network layer in the denoising network with at least one of the textual feature information of the prompt word and the second image feature information.

[0064] Figure 15 is a schematic diagram of a specific denoising network;

[0065] Figure 16 is a schematic diagram of generating a third liveness attack image using a decoder;

[0066] Figure 17 is a schematic flowchart of an image processing method provided in an embodiment of this application;

[0067] Figure 18 is a schematic diagram of an image generation process;

[0068] Figure 19 is a flowchart illustrating a training sample generation method provided in an embodiment of this application;

[0069] Figure 20 is a schematic diagram of a training sample generation process;

[0070] Figure 21 is a schematic block diagram of an image processing apparatus provided in an embodiment of this application;

[0071] Figure 22 is a schematic block diagram of a model training device provided in an embodiment of this application;

[0072] Figure 23 is a schematic block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0074] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. In embodiments of the invention, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0075] The image processing method provided in this application can be applied to various fields such as AI drawing and model training. When generating a first live attack image based on a first biometric image, it uses prompt words and second image feature information representing the image features of the target live attack as control information to process the first biometric image and generate the first live attack image. This ensures that the generated first live attack image not only meets the requirements of the prompt words but also satisfies the image features of the target live attack, thereby improving the generation quality of the first live attack image.

[0076] To facilitate understanding of the embodiments of this application, the relevant concepts involved in the embodiments of this application will first be introduced:

[0077] Image-to-image (IPE) refers to transforming the features of objects in an initial image according to input prompts to generate a target image that meets the requirements of the prompts. Currently, IPE methods are usually based on image generation models using diffusion algorithms, specifically transforming the initial image according to prompts input by the object (e.g., a user).

[0078] The image processing described in this application can be applied to scenarios where any image is generated, that is, to scenarios where another image is generated based on one image.

[0079] In some embodiments, the image processing method of this application can be used to generate training samples. For example, to generate training samples for a liveness detection model. Liveness detection technology is applied in various fields, such as intelligent facial recognition payment and identity authentication. Due to the complexity and variety of liveness attack types, relying solely on manual data collection and labeling cannot meet actual business needs. The collected data may be homogeneous and limited, failing to construct effective negative samples (i.e., bad cases), resulting in incomplete training of the liveness detection model, reduced detection accuracy, and potential risks such as infringement. Currently, methods for generating liveness attack samples mainly include those based on traditional GANs (Generative Adversarial Networks) and those based on diffusion models.

[0080] The main idea behind traditional GAN-based live attack sample generation techniques is to randomly initialize a Gaussian noise RGB image, input this Gaussian noise image into the generator's encoder to obtain a corresponding encoded feature vector, and then input the feature vector into the generator's decoder to obtain an image of the same size and dimensions as the original Gaussian noise RGB image through upsampling, denoted as the third image. Next, this third image and a real live attack image (such as a 3D mask, head mold, paper replica, or screen replica image) are input together into a binary classification network for discrimination. The generator strives to make the third image as similar to a real live attack image as possible, while the discriminator assigns a false label to the generated image (i.e., the third image) and a true label to the real live attack image. Through this continuous adversarial training, the generator can eventually generate results similar to real live attack images from any input Gaussian noise image. However, the overall performance of traditional GAN-based live attack sample generation techniques is poor, and the quality of the generated third images is low.

[0081] Among them, the liveness attack sample generation technique based on the diffusion model generates corresponding graphs by using text information as a control condition. However, the liveness attack samples generated by this technique are not realistic enough and are of poor quality, thus failing to meet the training requirements and resulting in poor model training performance.

[0082] In other words, current image generation technologies (such as live attack sample generation technology based on traditional GANs and live attack sample generation technology based on diffusion models) produce live attack images of poor quality that fail to meet requirements.

[0083] To address the aforementioned technical problems, this application proposes an image processing method. This method involves acquiring a first biometric image and a prompt word, where the prompt word indicates a preset live attack type. The first biometric image is a biometric image. Image feature extraction is performed on the first biometric image to obtain first image feature information, and text feature extraction is performed on the prompt word to obtain text feature information. Furthermore, second image feature information corresponding to the preset live attack type is determined. Then, using the text feature information of the prompt word and the second image feature information as control conditions, an image generation model is used to process the first image feature information of the first biometric image to obtain a first live attack image. Therefore, this application, when generating a first live attack image based on a first biometric image and a prompt word, uses the second image feature information corresponding to the preset live attack type and the prompt word as control conditions to process the first biometric image and generate the first live attack image. This ensures that the generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word but also includes the image feature information corresponding to the preset live attack type, thereby improving the generation quality of the first live attack image. If the above image processing method is applied to generate training data for a liveness detection model, realistic and high-quality liveness attack samples can be generated. When the liveness detection model is trained using these realistic and high-quality liveness attack samples, the accuracy of the liveness detection model in recognizing realistic liveness attack data can be improved, thereby enhancing the detection performance of the liveness detection model.

[0084] The implementation environment of the image processing method provided in the embodiments of this application is described below.

[0085] Figure 1 is a schematic diagram of the implementation environment of an image processing method provided in an embodiment of this application. As shown in Figure 1, the implementation environment includes a terminal device 101 and a server 102.

[0086] The terminal device 101 is connected to the server 102 via wired or wireless means.

[0087] In some embodiments, a client of an image processing system is installed on the terminal device 101, and the server 102 can be understood as the server-side or back-end of the image processing system. Objects (e.g., users) can interact with the client of the image processing system installed on the terminal device 101. The server 102 stores an image generation model; for example, this image generation model can be an image generation model based on a diffusion algorithm.

[0088] In some embodiments, before processing the image, the image generation model is first trained. The process of training the image generation model using server 102 is described below. Specifically, an object can send a prompt word to server 102 via terminal device 101. This prompt word indicates control information for generating a first live attack image based on a first biometric image, such as indicating a preset live attack type. Server 102 retrieves training images from a database, or terminal device 101 sends training images to server 102. Next, server 102 extracts image features from the training images to obtain first image feature information, and simultaneously extracts text features from the prompt word to obtain text feature information. Server 102 also presets second image feature information corresponding to the live attack type. Then, server 102 uses the text feature information of the prompt word and the second image feature information as control conditions, processes the first image feature information of the training images through the image generation model, and obtains a third live attack image. Based on the third live attack image and the second image feature information, the loss of the image generation model is determined, and the image generation model is trained based on the loss. For example, server 102 presets the difference between the second image feature information corresponding to the liveness attack type and the liveness attack feature information of the third liveness attack image, and determines the loss of the image generation model based on the difference.

[0089] In some embodiments, the image generation model described above can also be trained by the terminal device 101, and this application embodiment does not limit this.

[0090] In this embodiment of the application, after the image generation model is trained, the trained image generation model is used to execute the image processing method of this embodiment of the application.

[0091] In some embodiments, the image processing method provided in this application is executed by a system consisting of a terminal device 101 and a server 102. For example, an object (e.g., a user) launches a client of the image processing system on the terminal device 101. The client displays an input interface for a first biometric image and an input box for a prompt word. The object inputs the first biometric image to be processed in the input interface and inputs a prompt word in the prompt word input box, where the prompt word indicates a preset live attack type, and the first biometric image is a biometric image. The terminal device 101 sends the first biometric image and the prompt word to the server 102. The server 102 extracts the first image feature information of the first biometric image and the text feature information of the prompt word. Simultaneously, the server 102 presets the second image feature information corresponding to the live attack type. Then, the server 102 uses the text feature information of the prompt word and the second image feature information as control conditions, and processes the first image feature information of the first biometric image through an image generation model to obtain the first live attack image. For example, server 102 includes an image generation model. Server 102 inputs the first image feature information of the first biometric image, the text feature information of the prompt word, and the second image feature information into the image generation model. Using the text feature information of the prompt word and the second image feature information as control information, server 102 processes the first image feature information of the first biometric image to generate a first live attack image. Finally, server 102 sends the generated first live attack image to terminal device 101, and terminal device 101 displays the first live attack image to the target. Therefore, it can be seen that in this embodiment, when generating a first live attack image based on a first biometric image and a prompt word, the second image feature information and the prompt word are used as control conditions to process the first biometric image to generate a first live attack image. The generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word but also includes the image feature information corresponding to the preset live attack type, thereby improving the generation quality of the first live attack image. If the above image processing method is applied to generate training data for a liveness detection model, realistic and high-quality liveness attack samples can be generated. When the liveness detection model is trained using these realistic and high-quality liveness attack samples, the accuracy of the liveness detection model in recognizing realistic liveness attack data can be improved, thereby enhancing the detection performance of the liveness detection model.

[0092] In some embodiments, the image processing method provided in this application is executed by a terminal device 101. In this case, the terminal device 101 can be any terminal device capable of providing image processing services. The server 102 can be any server capable of providing data download services. In one example, the server 102 stores an image generation model. A client for the image processing system is installed on the terminal device 101, which can request the image generation model from the server 102 based on an object's request. For example, the terminal device 101 displays an input interface for a first biometric image and an input box for prompts. The object inputs the first biometric image to be processed in the input interface and a prompt in the prompt input box. In response to the input of the first biometric image and / or the prompt, the terminal device 101 sends a request to the server 102 to request the image generation model. The terminal device 101 performs image feature extraction on the first biometric image to obtain first image feature information of the first biometric image, and performs text feature extraction on the prompt to obtain text feature information of the prompt. Simultaneously, the terminal device 101 presets second image feature information corresponding to the liveness attack type. Then, using the textual feature information of the prompt word and the second image feature information as control conditions, the first image feature information of the first biometric image is processed by the image generation model to generate the first live attack image. The generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word, but also includes the image features of the target live attack under the preset live attack type.

[0093] In some embodiments, the terminal device 101 includes, but is not limited to, desktop computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle systems. Portable wearable devices may include smartwatches, smart bracelets, and head-mounted devices. Terminal devices are often equipped with a display device, which may be a monitor, display screen, touchscreen, etc., and the touchscreen may be a touchscreen, touch panel, etc.

[0094] In some embodiments, the server 102 may be one or more servers. When there are multiple servers 102, at least two servers may be used to provide different services, and / or at least two servers may be used to provide the same service, such as providing the same service in a load-balanced manner. This embodiment of the application does not limit this. The server 102 may be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms. The server 102 may also be a node in a blockchain.

[0095] It should be noted that the implementation environment of this application embodiment includes, but is not limited to, the one shown in FIG1.

[0096] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0097] First, referring to Figure 2, the training process of the image generation model in this application embodiment will be introduced.

[0098] Figure 2 is a schematic flowchart of a model training method provided in an embodiment of this application. The execution subject of this embodiment can be a device with model training functionality, such as a model training apparatus. This model training apparatus can be the server 102 or terminal device 101 shown in Figure 1, or it can be a system composed of the server 102 or terminal device 101 shown in Figure 1. For ease of description, the following embodiments use an electronic device as an example to illustrate the method of this embodiment.

[0099] As shown in Figure 2, the model training method of this application embodiment includes:

[0100] S101, The electronic device acquires training images and prompt words.

[0101] The prompt indicates the preset live attack type, and the training image is either a biometric image or a live attack image.

[0102] It should be noted that the data used in the embodiments of this application and the process of obtaining this data comply with the relevant laws and regulations.

[0103] In the field of image generation, prompts can be used to describe the requirements for image generation. For example, as shown in Figure 3, a prompt is used to instruct the generation of a kitten. Thus, in the field of image generation, the kitten shown on the right side of Figure 3 can be generated from the puppy in the image on the left side of Figure 3 based on this prompt.

[0104] The image generation model of this application embodiment can be applied to different fields to generate images that meet different requirements.

[0105] In one example, the image generation model of this application embodiment can be applied to the field of AI drawing to generate images that meet the requirements.

[0106] In one example, the image generation model of this application embodiment can be applied to the field of training sample generation to generate training samples that meet the requirements. For instance, this image generation model can be used to generate training data for a liveness detection model, such as to generate realistic, high-quality liveness attack samples.

[0107] In this embodiment of the application, when the image generation model is used in different fields, the image generation model can be trained using training images of the corresponding fields.

[0108] In this embodiment, when the image generation model is used in the field of liveness attack detection, biometric images or liveness attack images can be used as training images to train the image generation model. The biometric images are real images of living organisms. The liveness attack images are non-real biometric images, such as mask images or reproduced images.

[0109] A liveness attack refers to the act of forging or simulating liveness features (such as a face) through technical means to deceive a facial recognition system and thereby achieve illegal purposes. Depending on the attack method, liveness attacks include different types. In this embodiment, the type of liveness attack can be understood as the method of liveness attack.

[0110] In some embodiments, the types of liveness attacks in this application may include 3D masks, paper flipping, screen flipping, 3D head models, and other attack types.

[0111] Among these, the 3D mask liveness attack refers to an attack method where attackers use 3D masks to deceive facial recognition systems. Specifically, attackers construct a highly realistic 3D mask, typically made of materials such as silicone and plastic, that closely replicates the facial features of the person being impersonated. By placing this 3D mask in front of a camera, attackers can bypass the facial recognition system's liveness detection, thereby impersonating someone else.

[0112] The "paper-based liveness detection" attack refers to an attack that uses a printed photo of the person being impersonated. Specifically, the attacker prints a photo of the person being impersonated and then uses this printed photo for liveness detection, thereby bypassing facial recognition system verification.

[0113] Screen capture liveness detection attacks refer to a method of spoofing attacks by capturing images of faces displayed on the screen of an electronic device. Specifically, attackers capture images of faces displayed on the screen and then use these images in a liveness detection system to bypass the system's liveness detection mechanism.

[0114] 3D head model liveness detection attacks bypass facial recognition systems by using highly realistic 3D head models to simulate real human faces. This attack method utilizes 3D printing technology and high-precision silicone molds to make the head model very close to a real human face in appearance and texture, thus deceiving facial recognition systems.

[0115] To improve the training accuracy of the image generation model, cue words were also obtained, which indicated the preset liveness attack type.

[0116] The liveness attack types in this application mainly include 3D masks, paper flipping, screen flipping, and 3D head models. Different liveness attack characteristics can be included for different liveness attack types. In this application embodiment, a preset liveness attack type can be indicated by a prompt word, such as indicating that the preset liveness attack type is a 3D mask or screen flipping.

[0117] In this embodiment, the electronic device can process the training image based on a preset liveness attack type indicated by a prompt word, and generate a liveness attack image corresponding to that preset liveness attack type. For example, if the preset liveness attack type indicated by the prompt word is a 3D mask, the electronic device processes the training image to generate a 3D mask image, which is a liveness attack image.

[0118] This application does not limit the specific method by which the electronic device acquires training images and prompts.

[0119] In one possible implementation, the prompt words are input by an object (e.g., a user). All or part of the training images are training data from a database. For example, if the electronic device is a terminal device, the object inputs the prompt words to that terminal device. As another example, if the electronic device is a server, the object inputs the prompt words on the terminal device, and the terminal device sends the input prompt words to the server.

[0120] In one possible implementation, the aforementioned prompts are input by the object, and all or part of the training images are also input by the object. For example, if the electronic device is a terminal device, the object inputs the training images and prompts to the terminal device. As another example, if the electronic device is a server, the object inputs the training images and prompts on the terminal device, and the terminal device sends the input training images and prompts to the server. Optionally, the aforementioned training images can be images captured by the object, or images obtained by the object through other means.

[0121] In one possible implementation, all or part of the training images are generated by an electronic device, while the prompts are input by the user. For example, if the electronic device is a terminal device, it receives an image generation instruction from the user, which instructs the generation of training images that meet preset requirements. The terminal device then generates the training images based on this instruction. As another example, if the electronic device is a server, the terminal device receives the image generation instruction from the user and sends it to the server. The server then generates the training images based on this instruction.

[0122] The electronic device obtains a training dataset consisting of training images and prompt words using the above method, and then uses the training dataset and prompt words to train the image generation model.

[0123] In this embodiment, the training process of the image generation model using each training image in the training dataset is basically the same. For ease of description, a single training image is used as an example. In this embodiment, the image generation model undergoes multiple rounds of training. In each round, a batch of training images is selected from the training dataset for training. In this embodiment, the training image can be understood as any single training image from any round of training data. It should be noted that in this embodiment, during the multiple rounds of training of the image generation model, the prompt words corresponding to each round of training can be different. For example, different prompt words can be input in each round of training, or each training image can correspond to a single prompt word. Optionally, the prompt words input in each round of training can be the same. This embodiment does not restrict the input of prompt words and can set them according to actual needs.

[0124] S102. The electronic device extracts the first image feature information of the training image and the text feature information of the prompt words.

[0125] In this embodiment of the application, after the electronic device obtains the training image and the corresponding prompt word based on the above steps, in order to achieve the mutual fusion between the image information of the training image and the semantic information of the prompt word, the electronic device first extracts the first image feature information of the training image and the text feature information of the prompt word.

[0126] In some embodiments, before extracting the first image feature information of the training image, the electronic device first preprocesses the training image to obtain a preprocessed training image.

[0127] This application does not limit the specific method of preprocessing the training images in its embodiments. For example, it may process the clarity, sharpness, etc. of the training images.

[0128] In one example, the electronic device crops the training image to match the size of the input image of the image generation model, based on the input image size of the image generation model.

[0129] In some examples, the electronic device performs image matting on the training image, extracting the object region from the first biometric image. Then, it resizes this object region to a first size. Next, the electronic device randomly crops the first-sized object region to a second size, resulting in the preprocessed training image. Here, the second size is the size of the input image required by the image generation model.

[0130] For example, assuming the image generation model requires an input image size of 224*224*3, to achieve this, all training images need to be resized to this scale. Furthermore, in this embodiment, the prompt word is used to indicate a preset liveness attack type. Based on this, the electronic device performs image cutout of object regions in the training image. For example, if the training image includes a face, a face detection tool can be used to detect faces in the training image, and then the face region is cut out from the training image based on the detection results. Next, the cut-out face region is resized to a first size, for example, to 256*256*3. Finally, the electronic device uses a random cropping method to obtain an input image of a second size (e.g., 224*224*3).

[0131] In this embodiment, the training image can be latent space feature extracted using an image feature encoder. The main purpose of this part is to represent the image features using high-level semantic features, which not only reduces the dimensionality of the original image space and thus improves the training speed, but also facilitates better interaction with subsequent conditional features (i.e., control information).

[0132] In some examples, the image feature encoder described above can be any pre-trained model.

[0133] In one example, the image feature encoder can also be trained end-to-end together with the image generation model.

[0134] This application does not limit the specific network structure of the image feature encoder in its embodiments.

[0135] In some embodiments, the image feature encoder described above includes one or more convolutional layers. The electronic device directly extracts features from the training image using the image feature encoder to obtain the first image feature information of the training image.

[0136] In some embodiments, to improve image processing speed and reduce computational workload, the electronic device can encode and compress the training images to obtain the first image feature information of the training images. That is, the image feature encoder at this time can be a feature extractor with encoding and compression capabilities. For example, the image feature encoder can be an encoder of a Variational Autoencoder (VAE) or an image feature encoder / decoder in a CLIP (Contrastive Language-Image Pre-Training) model. In this way, the electronic device can encode and compress the training images using the encoder of a VAE or the image feature encoder / decoder in a CLIP model to obtain the first image feature information of the training images.

[0137] For example, as shown in Figure 4, the electronic device uses a VAE to encode and compress the training image to obtain the first image feature information of the training image. Specifically, the encoder in the VAE encodes the training image to transform it from pixel space to latent space, thereby obtaining the first image feature information of the training image.

[0138] For example, a training image is a 512*512 pixel photograph, represented by data as (3, 512, 512), which is the image data size of RGB three channels * 512*512 pixels, with a data volume of 786432. After VAE encoding, it is converted to the latent space, for example, compressed into image feature information of (4, 64, 64) in the latent space, with a data volume of 16384, reducing the data volume to 1 / 64th of the original. This can greatly improve the processing efficiency and universality of the image processing method provided in the embodiments of this application.

[0139] This application does not limit the specific method by which electronic devices extract text feature information of prompt words.

[0140] In some embodiments, the electronic device extracts text features from the prompt words using a text encoder to obtain the text feature information of the prompt words. For example, the text encoder includes an embedding layer.

[0141] In some embodiments, as shown in Figure 5, the electronic device extracts features from the prompt words using the text encoder in CLIP to obtain the text feature information of the prompt words. The CLIP text encoder converts the text information of the prompt words into numerically expressed information so that the machine can understand the semantics of the prompt word text. For example, the CLIP text encoder can be understood as a special Transformer language model; its input is text, and its output is a matrix of numerical representations. Specifically, the extracted words are divided into multiple tokens (tokens or characters), and the embedding vector of each token is extracted. The embedding vectors of multiple tokens are combined to form an embedding matrix, which is recorded as the text feature information of the prompt word.

[0142] In addition to using the above method to extract the first image feature information of the training image and the text feature information of the prompt words, electronic devices can also use other methods to obtain them, and this application embodiment does not limit this.

[0143] After the electronic device extracts the first image feature information of the training image and the text feature information of the prompt words, it performs the following step S104.

[0144] S103. The electronic device determines the second image feature information corresponding to the preset liveness attack type.

[0145] It should be noted that there is no specific order in the execution of S102 and S103. That is to say, S102 can be executed before S103, after S103, or simultaneously with S103.

[0146] The focus of this application's embodiments is how to train a more refined image generation model. As mentioned above, relying solely on text (i.e., prompts) as conditions to guide the image generation model cannot achieve the desired results. In this application's embodiments, when training the image generation model using training images, to improve the accuracy of generating liveness attack images, in addition to obtaining prompts, second image feature information corresponding to a preset liveness attack type is determined. This second image feature information can be understood as the main image difference information between the image corresponding to the preset liveness type and the image of a real liveness attack. For example, for the 3D mask attack type, since there is a significant difference in material between mask texture and real human texture, texture is an image feature that meets the requirements for liveness detection. Therefore, texture feature information is determined as the image feature information corresponding to the 3D mask attack type. Thus, when this application's embodiments use the second image feature information as an auxiliary condition to train the image generation model, the trained image generation model can generate images that conform to the second image feature information corresponding to the preset liveness attack type.

[0147] The specific methods for determining the second image feature information corresponding to the preset liveness attack type in this application embodiment include, but are not limited to, the following:

[0148] Method 1: Determine N candidate image feature information corresponding to a preset liveness attack type, and select one candidate image feature information from these N candidate image feature information as the second image feature information, where N is a positive integer. The N candidate image feature information represent the image feature information of N types of liveness attacks belonging to the preset liveness attack type.

[0149] The liveness attack types in this application include 3D masks, paper flipping, screen flipping, 3D head models, and other attack types. In this application, different liveness attack types may include different image feature information for the liveness attack.

[0150] For example, 3D masks and 3D head models differ significantly in material from real living beings, and different materials have different texture characteristics. Therefore, the image feature information for liveness attacks using 3D masks and 3D head models can include image feature information such as texture.

[0151] For example, compared to real living objects, photographs of paper or screens can exhibit features such as shadows and image edges. Therefore, the image feature information for liveness attacks using photographs of paper or screens can include image features such as shadows and image edges.

[0152] In this first method, the electronic device determines N candidate image feature information corresponding to a preset liveness attack type based on the above method. The electronic device selects one candidate image feature information from these N candidate image feature information as the second image feature information. For example, the electronic device may select a default candidate image feature information as the second image feature information. Alternatively, the electronic device may display these N candidate image feature information to an object, allowing the object to select one candidate image feature information from these N candidate image feature information as the second image feature information.

[0153] For example, suppose the liveness attack type is a 3D mask. This 3D mask can include different materials, such as N different materials. The electronic device determines the image feature information corresponding to these N different materials as N candidate image feature information for the 3D mask. In one example, the electronic device displays the image feature information corresponding to these N different materials to the object, and the object selects the image feature information corresponding to the silicone material from these N different materials. Based on the object's selection of the silicone material's image feature information, the electronic device uses this silicone material's image feature information as the second image feature information. In another example, the electronic device displays these N different materials to the object, and the object selects the silicone material from these N different materials, thus using the silicone material as the selected reference feature. Based on the object's selection of the silicone material, the electronic device extracts the image feature information of the silicone material as the second image feature information. Optionally, the image feature information corresponding to the aforementioned material can be the material's image texture feature information.

[0154] Method 2: If the training image is a live attack image, then the live attack feature information of the training image is extracted as the second image feature information.

[0155] In this second method, if the training image is a live attack image, it means that the training image is a live attack image corresponding to the preset live attack type, and the electronic device can directly determine the live attack feature information of the training image as the second image feature information.

[0156] For example, when the training image is a screenshot, the electronic device extracts the edge feature information of the training image as the second image feature information.

[0157] For example, when the training image is a 3D mask or 3D head model, the electronic device extracts the texture feature information of the training image as the second image feature information.

[0158] This application does not limit the specific method by which an electronic device extracts texture feature information from training images. For example, texture feature information from training images can be extracted using methods such as gray-level co-occurrence matrix, Tamura texture features, autoregressive texture models, and wavelet transform.

[0159] In this second method, the live attack image is used as the training image, and the live attack feature information of the training image is extracted as the second image feature information. This can obtain the image information corresponding to the preset live attack type, improve the training accuracy of the model, and reduce the difficulty of obtaining the second image feature information.

[0160] Method 3: Based on a preset live attack type, obtain a fourth live attack image, extract the live attack feature information of the fourth live attack image, and use it as the second image feature information.

[0161] In this third method, when training the image generation model based on the training images, a fourth liveness attack image is also used. This fourth liveness attack image can be a liveness attack image corresponding to a preset liveness attack type input by the object, or a liveness attack image corresponding to a preset liveness attack type downloaded by the electronic device from the network. This application embodiment does not limit the specific method of obtaining the fourth liveness attack image. The electronic device can extract the liveness attack feature information of the fourth liveness attack image as the second image feature information. It should be noted that if the above-mentioned training image is a liveness attack image, the liveness attack type of the fourth liveness attack image can be the same as or different from the liveness attack type of the training image; this application embodiment does not limit this. The specific method of extracting the liveness attack feature information of the fourth liveness attack image can refer to the relevant method of extracting the liveness attack feature information of the training image in the above-mentioned method two, and this application embodiment will not elaborate further.

[0162] In this third method, by using the live attack feature information of the fourth live attack image as the second image feature information, accurate image information corresponding to the preset live attack type can be obtained, improving the training accuracy of the model. It can also reduce the difficulty of obtaining the second image feature information, enrich the types of live attack image information, and thus improve the generalization ability of the model.

[0163] In some embodiments, the aforementioned N candidate image feature information, or the live attack feature information of the training image, or the live attack feature information of the fourth live attack image, are image texture feature information.

[0164] S104. The electronic device inputs the text feature information of the prompt word, the first image feature information, and the second image feature information into the image generation model. The image generation model is configured to generate a third liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0165] In this embodiment, to ensure that the image generated by the image generation model not only meets the requirements of the preset liveness attack type indicated by the prompt, but also possesses the image detail features corresponding to the preset liveness attack type, the first image feature information of the training image and the text feature information of the prompt are extracted during image generation model training. Furthermore, the second image feature information corresponding to the preset liveness attack type is determined. As shown in Figure 6, the first image feature information of the training image, the text feature information of the prompt, and the second image feature information are then input into the image generation model. The image generation model uses the text feature information of the prompt and the second image feature information as control conditions to process the first image feature information of the training image, generating a third liveness attack image. Because the generation of the third liveness attack image considers both the second image feature information and the text feature information of the prompt, the generated third liveness attack image not only meets the requirements of the preset liveness attack type indicated by the prompt, but also includes the second image feature information corresponding to the preset liveness attack type, thereby improving the effect of the third liveness attack image generated by the model.

[0166] The embodiments of this application do not limit the specific network structure of the above image generation model.

[0167] In some embodiments, as shown in FIG7, the image generation model includes a denoising network, and in this case, S104 includes the following steps S104-A to S104-C:

[0168] S104-A: The electronic device adds noise to the first image feature information of the training image to obtain the noise-added feature information of the training image.

[0169] S104-B: The electronic device uses a denoising network to denoise the noisy feature information of the training image based on the text feature information of the prompt words and the second image feature information, and obtains the denoised image feature information.

[0170] The S104-C electronic device decodes the denoised image feature information to obtain the third liveness attack image.

[0171] In this implementation, forward diffusion and backward diffusion techniques are used to generate a third liveness attack image. The forward diffusion process adds noise to the image, gradually transforming it into a featureless, noisy image. For example, the forward process would turn any image of a cat or dog into a noisy image, making it indistinguishable between a dog and a cat. Backward diffusion progressively estimates the noise and subtracts it from the noisy image, ultimately yielding an image of a cat or a dog.

[0172] In this embodiment, the electronic device, based on the above steps, obtains the first image feature information and the second image feature information of the training image, as well as the text feature information of the prompt word. Then, it performs noise enhancement processing on the first image feature information of the training image, for example, adding a preset noise value to the first image feature information of the training image to obtain the noisy feature information of the training image. For example, Gaussian noise can be gradually added to the first image feature information of the training image to obtain the noisy feature information of the training image. This process can be understood as a forward diffusion process.

[0173] Next, the electronic device performs denoising on the noisy feature information of the training image based on the second image feature information and the text feature information of the prompt words, and obtains the denoised image feature information. This process can be understood as a reverse diffusion process.

[0174] This application embodiment does not limit the specific process by which an electronic device performs denoising processing on the noise-added feature information of a training image based on the second image feature information and the text feature information of the prompt words, to obtain the denoised image feature information.

[0175] In some embodiments, the electronic device uses mathematical techniques to perform multiple denoising processes on the noisy feature information of the training image, using the textual feature information of the prompt words and the second image feature information as control conditions, to obtain the denoised image feature information.

[0176] In some embodiments, the electronic device uses a denoising network to perform multiple denoising processes on the noisy feature information of the training image, using the textual feature information of the prompt and the second image feature information as control conditions. The denoising network predicts the noise value for each denoising process. Specifically, during each denoising step, the noise value predicted by the denoising network is subtracted from the previous denoised feature information, generating new denoised feature information which is then input into the denoising network for further denoising. This process is repeated multiple times until all the noise added to the training image is removed. During the denoising process, in addition to the new denoised feature information of the training image and the textual feature information of the prompt, the denoising network also inputs the second image feature information. The image feature information is brought closer to the textual feature information of the prompt and the second image feature information during denoising, so that the final denoised image feature information incorporates both the textual feature information of the prompt and the second image feature information. Thus, the third liveness attack image generated based on the denoised image feature information not only meets the requirements of the preset liveness attack type indicated by the prompt but also includes the second image feature information corresponding to the preset liveness attack type, thereby improving the image generation effect of the image generation model.

[0177] As shown in Figure 8, the denoising process using a denoising network is an iterative process. For example, the denoising network uses at least one of the textual features of the prompt word and the second image features as control conditions to denoise the i-th denoised feature information of the training image, obtaining the i-th noise prediction value. Then, based on the i-th noise prediction value and the i-th denoised feature information of the training image, the (i+1)-th denoised feature information of the training image is obtained. Next, the (i+1)-th denoised feature information is used as the new i-th denoised feature information. The denoising network then uses at least one of the textual features of the prompt word and the second image features as control conditions to denoise the new i-th denoised feature information, obtaining the i-th noise prediction value. Based on the i-th noise prediction value and the i-th denoised feature information of the training image, the (i+1)-th denoised feature information of the training image is obtained. Next, the denoising feature information from the (i+1)th denoising step is used as the new denoising feature information from the ith step, and the process is repeated N times to obtain the denoising feature information from the Nth step. This denoising feature information from the Nth step is then used as the denoised image feature information. It should be noted that in these N denoising steps, at least one step uses the text feature information of the prompt word as the control condition, and at least one step uses the second image feature information as the control condition. In other words, while the overall control conditions for these N denoising steps are the text feature information of the prompt word and the second image feature information, in a specific denoising step, the control conditions may be either the text feature information of the prompt word, the second image feature information, or a combination of both.

[0178] In some embodiments, during each denoising process, the textual feature information of the prompt word and the second image feature information are used as control conditions. In this case, S104-B includes the following steps S104-B1 to S104-B3:

[0179] S104-B1. The electronic device uses a denoising network to denoise the i-th denoising feature information of the training image based on the text feature information of the prompt word and the second image feature information, and obtains the i-th noise prediction value. i is a positive integer less than or equal to N. If i is 1, the i-th denoising feature information is the noise-added feature information of the training image.

[0180] S104-B2, the electronic device obtains the (i+1)th denoising feature information of the training image based on the i-th noise prediction value and the i-th denoising feature information of the training image;

[0181] S104-B3: The electronic device takes the (i+1)th denoising feature information as the new ith denoising feature information, returns and repeats the above steps to obtain the Nth denoising feature information of the training image.

[0182] S104-B4 The electronic device determines the Nth denoising feature information as the denoised image feature information.

[0183] For example, assuming N equals 20, as shown in Figure 9, the electronic device first inputs the noise-added feature information of the training image, the text feature information of the prompt word, and the second image feature information into the denoising network for denoising, obtaining the first noise prediction value preset by the denoising network. Based on this first noise prediction value and the noise-added feature information of the training image, the first denoising feature information of the training image is obtained. Next, the first denoising feature information of the training image, the text feature information of the prompt word, and the second image feature information are input into the denoising network for denoising, obtaining the second noise prediction value preset by the denoising network. Based on this second noise prediction value and the first denoising feature information, the second denoising feature information is obtained. Next, the second denoising feature information of the training image, the text feature information of the prompt word, and the second image feature information are input into the denoising network for denoising, obtaining the third noise prediction value preset by the denoising network. Based on this third noise prediction value and the second denoising feature information, the third denoising feature information is obtained. The process is repeated, inputting the 19th denoising feature information of the training image, the text feature information of the prompt word, and the second image feature information into the denoising network for denoising, to obtain the 20th noise prediction value preset by the denoising network. Based on this 20th noise prediction value and the 19th denoising feature information, the 20th denoising feature information is obtained. This 20th denoising feature information is then determined as the denoised image feature information.

[0184] The following describes the specific process in S104-B1 above, in which the textual feature information of the prompt word and the second image feature information are used as control conditions, and the i-th noise prediction value is obtained by denoising the i-th denoising feature information of the training image through a denoising network.

[0185] In this application embodiment, the electronic device uses the text feature information of prompt words and the second image feature information as control conditions. The denoising network is used to denoise the i-th denoising feature information of the training image to obtain the i-th noise prediction value. The specific method is not limited.

[0186] In some embodiments, the denoising network includes multiple network layers (e.g., M network layers). During the i-th denoising process, the electronic device uses the textual feature information of the prompt word and the second image feature information as control conditions. When the denoising network processes the i-th denoising feature information of the training image, at least one of the textual feature information of the prompt word and the second image feature information can be fused with the output feature information of at least some network layers in the denoising network. This achieves the binding of the textual feature information of the prompt word and the second image feature information as conditions and the distribution of the training image. For example, the textual feature information of the prompt word can be fused with the output feature information of some network layers in the denoising network, and the second image feature information can be fused with the output feature information of other network layers in the denoising network. Another example is fusing the textual feature information of the prompt word with the output feature information of some network layers in the denoising network, and fusing the second image feature information with the output feature information of all network layers in the denoising network. Yet another example is fusing the second image feature information with the output feature information of some network layers in the denoising network, and fusing the textual feature information of the prompt word with the output feature information of all network layers in the denoising network. For example, the second image feature information is fused with the output feature information of all network layers in the denoising network, and the text feature information of the prompt word is fused with the output feature information of all network layers in the denoising network.

[0187] In some embodiments, S104-B1 above includes the following steps S104-B11 and S104-B12:

[0188] S104-B11, The electronic device uses the i-th denoising feature information of the training image as the input feature information of the denoising network;

[0189] S104-B12. The electronic device fuses the feature information output by the first part of the denoising network layer with the text feature information of the prompt word at least, and fuses the feature information output by the second part of the denoising network layer with the second image feature information at least, to obtain the i-th noise prediction value.

[0190] During the i-th denoising process, the electronic device acquires the i-th denoising feature information of the training image obtained in the (i-1)-th denoising process, and inputs the i-th denoising feature information of the training image into the denoising network.

[0191] In some embodiments, when the i-th denoising feature information of the training image is used as the input feature information of the denoising network in S104-B11, the electronic device may also fuse the i-th denoising feature information of the training image with at least one of the text feature information of the prompt word and the second image feature information, and use the fused feature information as the input feature information of the first network layer of the denoising network.

[0192] As shown in Figure 10, the denoising network includes multiple network layers. The electronic device fuses the feature information output by the first part of the denoising network layer with at least the text feature information of the prompt word, and fuses the feature information output by the second part of the denoising network layer with at least the second image feature information to obtain the i-th noise prediction value.

[0193] Optionally, there are no overlapping network layers between the first part of the network layer and the second part of the network layer.

[0194] Optionally, the first part of the network layer and the second part of the network layer may include overlapping network layers.

[0195] Optionally, one or more other network layers may be included between the first part of the network layer and the second part of the network layer, that is, the first part of the network layer and the second part of the network layer are not continuous.

[0196] Optionally, the first part of the network layer (or the second part of the network layer) can be a continuous network layer in the denoising network, or it can be a discontinuous network layer.

[0197] S104-B12 above fuses the feature information output by the first part of the denoising network layer with at least the text feature information of the prompt word, and fuses the feature information output by the second part of the denoising network layer with at least the second image feature information to obtain the i-th noise prediction value, including at least the cases shown in the following examples:

[0198] In one example, as shown in Figure 11, the electronic device fuses the feature information output from each layer of the denoising network with the textual feature information of the prompt word and the second image feature information, and uses the fused feature information as the input feature information for the next network layer. This process continues until the feature information output from the last network layer of the denoising network is obtained, which is used as the i-th noise prediction value. In this example, the first and second part of the network layers are the same, both consisting of M network layers in the denoising network.

[0199] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with the text feature information of the prompt word, and fuses the output feature information of the second part of the denoising network layer with the second image feature information. The first and second part of the network layer do not contain the same network layers. For example, as shown in Figure 12, the electronic device fuses feature information 1 output from the first network layer of the denoising network with the text feature information of the prompt word, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is then fused with the second image feature information and used as the input feature information of the third network layer, and so on, to obtain the feature information output from the last network layer of the denoising network, which is used as the i-th noise prediction value.

[0200] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with at least one of the text feature information of the prompt word and the second image feature information. Similarly, it fuses the output feature information of the second part of the denoising network layer with both the text feature information of the prompt word and the second image feature information. The first and second part of the network layer do not contain the same network layers. For example, as shown in Figure 13, the electronic device fuses feature information 1 output from the first network layer of the denoising network with the text feature information of the prompt word (or with the second image feature information), and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is then fused with the second image feature information (or the text feature information of the prompt word) and used as the input feature information of the third network layer. The output feature information 3 of the third network layer is then fused with both the second image feature information and the text feature information of the prompt word and used as the input feature information of the fourth network layer. This process continues until the feature information output by the last network layer of the denoising network is obtained, which is used as the i-th noise prediction value.

[0201] This application does not limit the specific method by which the output feature information of some or all layers of the denoising network is fused with the text feature information of the prompt word and at least one of the second image feature information. It only requires ensuring that the denoising network considers both the second image feature information and the text feature information of the prompt word when denoising the i-th denoising feature information of the training image.

[0202] In this embodiment, to facilitate the description of the process of fusing the output feature information of the network layer in the denoising network with the text feature information of the prompt word and at least one of the second image feature information, the k-th network layer in the denoising network is used as an example for explanation. In this case, S104-B12 may include the following steps S104-B12-a and S104-B12-b:

[0203] S104-B12-a. For the kth network layer in the denoising network, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network, where k is a positive integer.

[0204] S104-B12-b: The electronic device obtains the i-th noise prediction value based on the input feature information of the (k+1)-th network layer.

[0205] This application does not limit the specific network structure of the denoising network. For example, it can be any neural network that predicts noise values ​​based on noisy image feature information and text feature information. For example, it can be a U-Net network, which, while removing noise from image information, diffuses the denoised image feature information in the direction of the preset liveness attack type indicated by the prompt word and the second image feature information corresponding to the preset liveness attack type, so as to generate the second image feature information that satisfies the preset liveness attack type indicated by the prompt word and the second image feature information corresponding to the preset liveness attack type.

[0206] In this embodiment, for the k-th layer of the denoising network, the electronic device fuses the output feature information of the k-th layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)-th layer of the denoising network. For example, as shown in Figure 14A, the electronic device fuses the output feature information of the k-th layer with the text feature information of the prompt word and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network. As another example, as shown in Figure 14B, the electronic device fuses the output feature information of the k-th layer with the second image feature information and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network. As yet another example, as shown in Figure 14C, the electronic device fuses the output feature information of the k-th layer with the second image feature information and the text feature information of the prompt word, and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network.

[0207] This application embodiment does not limit the specific method by which the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network.

[0208] In some embodiments, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information through one or more convolutional layers to obtain the input feature information of the (k+1)th network layer in the denoising network.

[0209] In some embodiments, the electronic device adds, multiplies, or concatenates the output feature information of the k-th network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain fused feature information. Then, based on this fused feature information, the input feature information of the (k+1)-th network layer is determined. For example, the fused feature information is determined as the input feature information of the (k+1)-th network layer. As another example, the electronic device performs other processing on the fused feature information (e.g., convolution) and uses it as the input feature information of the (k+1)-th network layer. In some embodiments, other processing such as convolution can be performed before or after at least one of the output feature information of the k-th network layer, the text feature information of the prompt word, and the second image feature information before fusion; this application embodiment does not limit this.

[0210] In some embodiments, the electronic device uses an attention mechanism to fuse the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer.

[0211] In one possible implementation, the textual feature information of the prompt word is used as the key value (K) of the attention module, the second image feature information is used as the representative value (V) of the attention module, and the output feature information of the kth network layer is used as the query value (Q) of the attention module. These are input into the attention module for fusion processing to obtain the fused feature information. Then, based on the fused feature information, the input feature information of the (k+1)th network layer is determined.

[0212] For example, electronic devices can perform information fusion using the following formula (1):

[0213] Among them, A k For the fused feature information, Q k K represents the output feature information of the k-th network layer in the denoising network. T V represents the textual features of the prompt words, and V represents the second image feature information. d is a scaling factor used to adjust the input scale of the softmax function.

[0214] In one possible implementation, the textual feature information of the prompt word is used as the representative value (V) of the attention module, the second image feature information is used as the key value (K) of the attention module, and the output feature information of the kth network layer is used as the query value (Q) of the attention module. These are input into the attention module for fusion processing to obtain the fused feature information. Then, based on the fused feature information, the input feature information of the (k+1)th network layer is determined.

[0215] The methods for determining the input feature information of the (k+1)th network layer based on the fused feature information include at least the following: Method 1, determining the fused feature information as the input feature information of the (k+1)th network layer; Method 2, processing the fused feature information (e.g., convolutional processing), and determining the processing result as the input feature information of the (k+1)th network layer.

[0216] The above describes the specific process by which an electronic device fuses the output feature information of the k-th network layer in the denoising network with at least one of the second image feature information and the text feature information of the prompt word, and uses this as the input feature information of the (k+1)-th network layer. Based on the above steps, the electronic device can determine the feature information output by the last network layer of the denoising network.

[0217] For example, as shown in Figure 15, suppose the denoising network includes 4 network layers and 3 attention modules. In the i-th denoising process, the electronic device inputs the i-th denoising feature information of the training image into the first network layer for processing, obtaining the output feature information 1 of the first network layer. Next, the output feature information 1 of the first network layer is fused with the text feature information of the prompt word and the second image feature information into the first attention module, obtaining the fused feature information 1 output by the first attention module. This fused feature information 1 is then used as the input feature information of the second network layer, processed in the second network layer of the denoising network, obtaining the output feature information 2 of the second network layer. Next, the output feature information 2 of the second network layer is fused with the text feature information of the prompt word and the second image feature information into the second attention module, obtaining the fused feature information 2 output by the second attention module. This fused feature information 2 is then used as the input feature information of the third network layer, processed in the third network layer of the denoising network, obtaining the output feature information 3 of the third network layer. Next, the output feature information 3 of the third network layer, along with the text feature information of the prompt word and the second image feature information, is input into the third attention module for fusion processing, resulting in the fused feature information 4 output by the third attention module. This fused feature information 4 is then used as the input feature information of the fourth network layer, and processed in the last network layer of the denoising network to obtain the output feature information 4 of the last network layer. Finally, based on the output feature information 4 of the last network layer, the i-th noise prediction value is obtained.

[0218] In one example, an electronic device can determine the i-th noise prediction value by using the feature information output by the last network layer of the denoising network.

[0219] Next, the steps S104-B2 above are executed to obtain the (i+1)th denoising feature information of the training image based on the i-th noise prediction value and the i-th denoising feature information of the training image.

[0220] For example, the difference between the i-th denoising feature information and the i-th noise prediction value of the training image is determined as the (i+1)-th denoising feature information of the training image.

[0221] The electronic device takes the (i+1)th denoising feature information of the training image determined above as the new ith denoising feature information, returns to execute the above steps S104-B1, repeats the execution N times, obtains the ith denoising feature information of the training image, and then uses the ith denoising feature information as the denoised image feature information.

[0222] Finally, the electronic device performs the steps of S104-C above, that is, the electronic device decodes the denoised image feature information to obtain the third liveness attack image.

[0223] In this embodiment of the application, the denoised image feature information is feature information in the latent space. The electronic device needs to decode the denoised image feature information to convert the denoised image feature information from the latent space to the pixel space to obtain the third liveness attack image.

[0224] This application does not limit the specific method by which an electronic device decodes the denoised image feature information to obtain a third liveness attack image. For example, the electronic device uses an image decoder to decode the denoised image feature information to obtain a third liveness attack image.

[0225] In some embodiments, as shown in FIG16, the electronic device decodes the denoised image feature information using a VAE decoder to obtain a third liveness attack image. This third liveness attack image is generated based on the prompt word and the second image feature information, ensuring that the generated third liveness attack image not only meets the requirements of the preset liveness attack type indicated by the prompt word, but also includes the image detail features corresponding to the preset liveness attack type, thereby improving the image generation effect.

[0226] It should be noted that the VAE decoder described above can be a pre-trained decoder. Optionally, the VAE decoder can be trained end-to-end together with the image generation model described above.

[0227] Based on the above steps, the electronic device processes the first image feature information, the text feature information of the prompt words, and the second image feature information of the training image using an image generation model to generate a third liveness attack image. Then, the following step S105 is executed.

[0228] S105. The electronic device determines the loss of the image generation model based on the second image feature information and the third liveness attack image, and trains the image generation model based on the loss.

[0229] In this embodiment, the electronic device processes the first image feature information, the text feature information of the prompt word, and the second image feature information of the training image using an image generation model to generate a third liveness attack image. Then, based on this third liveness attack image and the second image feature information, the loss of the image generation model is determined. Furthermore, based on this loss, the image generation model is trained.

[0230] This application does not limit the specific method by which the electronic device determines the loss of the image generation model based on the third liveness attack image and the feature information of the second image.

[0231] In some embodiments, it is desirable that the live attack feature information of the third live attack image recovered by the image generation model remains consistent with the feature information of the second image. Based on this, S105 includes the following steps S105-A and S105-B:

[0232] S105-A: The electronic device extracts live attack features from the third live attack image to obtain the live attack feature information corresponding to the third live attack image.

[0233] S105-B, the electronic device determines the loss of the image generation model based on the second image feature information and the live attack feature information of the third live attack image.

[0234] In this implementation, when the electronic device generates the loss of the image generation model based on the third liveness attack image and the feature information of the second image, it extracts liveness attack features from the third liveness attack image to obtain the liveness attack feature information corresponding to the third liveness attack image. Then, based on the liveness attack feature information of the third liveness attack image and the feature information of the second image, the loss of the image generation model is determined.

[0235] This application embodiment does not limit the specific method by which the electronic device determines the loss of the image generation model based on the liveness attack feature information and the second image feature information of the third liveness attack image. For example, the corresponding liveness attack feature information and the second image feature information of the third liveness attack image can be substituted into a preset loss function to calculate the loss of the image generation model. This application embodiment does not limit the specific type of the preset loss function, such as a cross-entropy loss function. As another example, the electronic device can determine the difference between the corresponding liveness attack feature information and the second image feature information of the third liveness attack image, and determine the model loss based on this difference.

[0236] In some embodiments, if the second image feature information is the texture feature information of the training image, then the above-mentioned extraction of liveness attack features from the third liveness attack image to obtain the liveness attack feature information of the third liveness attack image includes: extracting texture features from the third liveness attack image to obtain the texture feature information of the third liveness attack image. Correspondingly, the above-mentioned determination of the loss of the image generation model based on the second image feature information and the liveness attack feature information of the third liveness attack image includes: determining the loss of the image generation model based on the texture feature information of the training image and the texture feature information of the third liveness attack image.

[0237] This application embodiment does not limit the specific method by which the electronic device determines the loss of the image generation model based on the texture feature information of the third liveness attack image and the texture feature information of the training image. For example, the texture feature information of the third liveness attack image and the texture feature information of the training image are substituted into a preset loss function to calculate the loss of the image generation model. This application embodiment does not limit the specific type of the preset loss function, for example, it can be a cross-entropy loss function, etc.

[0238] In some embodiments, the electronic device determines the difference between the texture feature information of the training image and the texture feature information of the third liveness attack image; based on this difference, it determines the loss of the image generation model.

[0239] In one example, the loss of the image generation model includes texture loss.

[0240] For example, an electronic device can use the following formula (2) to determine the texture loss L. mse L mse =||f ori -f rec ||2 (2)

[0241] Among them, f ori This represents the texture feature information of the training image, while f rec This represents the texture feature information of the third liveness attack image. || ||2 is the Euclidean distance.

[0242] In one example, the loss of the image generation model in this embodiment of the application also includes a diffusion loss. This diffusion loss can be calculated using the MSE loss function. This embodiment of the application does not limit this.

[0243] Based on the steps described above, the electronic device determines the loss of the image generation model and updates the parameters of the model based on this loss. Simultaneously, it determines whether the training termination condition of the image generation model is met. The training termination condition can be that the model's loss meets a preset loss, or the training iterations reach a preset number. If the training termination condition is not currently met, a new batch of training samples is selected, and the updated image generation model is retrained using the same method. This process is repeated until the training termination condition is met, ultimately resulting in the trained image generation model.

[0244] The model training method provided in this application involves acquiring a training image and a prompt word, where the prompt word indicates a preset live attack type. The training image is either a biometric image or a live attack image. Next, image feature extraction is performed on the training image to obtain first image feature information, and text feature extraction is performed on the prompt word to obtain text feature information. Furthermore, second image feature information corresponding to the preset live attack type is determined. Then, the text feature information of the prompt word, the first image feature information, and the second image feature information are input into an image generation model. Based on the first image feature information of the training image, the image generation model generates a third live attack image that conforms to the text feature information and the second image feature information. Based on the training image and the third live attack image, the loss of the image generation model is determined, and the image generation model is trained based on this loss. Therefore, in this embodiment, when generating a third liveness attack image based on training images and prompts, the second image feature information and prompts are used as control conditions to process the training images and generate the third liveness attack image. The generated third liveness attack image not only satisfies the preset liveness attack type indicated by the prompts but also includes the image detail features corresponding to the preset liveness attack type, thereby improving the generation quality of the third liveness attack image generated by the model. Furthermore, in this embodiment, the loss of the image generation model is determined based on the second image feature information and the third liveness attack image, enabling the image generation model to focus on the image detail features of the liveness attack, thereby improving the image generation performance of the image generation model.

[0245] The training process of the image generation model has been introduced above. The image processing process based on the trained image generation model will be introduced below.

[0246] Figure 17 is a schematic flowchart of an image processing method provided in an embodiment of this application. The executing entity of this embodiment can be a device with image processing capabilities, such as an image processing apparatus. This image processing apparatus can be the server 102 or terminal device 101 shown in Figure 1, or it can be a system composed of the server 102 or terminal device 101 shown in Figure 1. For ease of description, the following embodiments use an electronic device as an example to illustrate the method of this embodiment.

[0247] As shown in Figure 17, the image processing method of this application embodiment includes:

[0248] S201, The electronic device acquires the first biometric image and prompt word.

[0249] This prompt indicates the preset liveness attack type.

[0250] The image processing method of this application embodiment can be applied to different fields to generate images that meet different requirements.

[0251] In one example, the image processing method of this application embodiment can be applied to the field of AI drawing to generate images that meet the requirements.

[0252] In one example, the image processing method of this application embodiment can be applied to the field of training sample generation to generate training samples that meet the requirements. For instance, this image processing method can be used to generate training data for a liveness detection model, such as to generate realistic, high-quality liveness attack samples.

[0253] In the embodiments of this application, when the image processing method is used in different fields, a first biometric image of the corresponding field can be obtained.

[0254] The first biometric image in this application embodiment can be understood as an image to be processed or modified. The first biometric image includes at least one object.

[0255] In this embodiment of the application, the prompt word is used to indicate the preset liveness attack type.

[0256] The liveness attack types in this application include 3D masks, paper flipping, screen flipping, 3D head models, and other attack types. In this application, different liveness attack characteristics may be included for different liveness attack types. In this application, a prompt word can be used to indicate the preset liveness attack type, such as indicating that the preset liveness attack type is a 3D mask or screen flipping.

[0257] This application does not limit the specific method by which the electronic device acquires the first biometric image and prompt words.

[0258] In one possible implementation, both the aforementioned prompt and the first biometric image are object inputs.

[0259] In one possible implementation, the first biometric image is generated by an electronic device, and the prompt is input by the object.

[0260] S202, The electronic device extracts the first image feature information and the text feature information of the prompt word from the first biometric image.

[0261] In this embodiment of the application, after the electronic device obtains the first biometric image and the prompt word corresponding to the first biometric image based on the above steps, in order to achieve the mutual fusion between the image information of the first biometric image and the semantic information of the prompt word, the electronic device first extracts the first image feature information of the first biometric image and the text feature information of the prompt word.

[0262] In some embodiments, before extracting the first image feature information of the first biometric image, the electronic device first preprocesses the first biometric image to obtain a preprocessed first biometric image.

[0263] The specific method of preprocessing the first biometric image in this application embodiment is not limited. For example, the clarity and sharpness of the first biometric image may be processed.

[0264] In one example, the electronic device crops the size of the first biometric image to match the size of the input image of the image generation model, based on the input image size of the image generation model.

[0265] In some examples, the electronic device performs image matting on the first biometric image, extracting object regions from the image. Then, it resizes these object regions to a first size. Next, the electronic device centers and crops the first-sized object regions to a second size, resulting in a preprocessed first biometric image. The second size is the size of the input image required by the image generation model.

[0266] Next, the electronic device extracts image features from the preprocessed first biometric image.

[0267] As shown in Figure 18, this embodiment of the application encodes the first biometric image using an image feature encoder to compress the first biometric image from the pixel space into the latent space, thereby obtaining the first image feature information of the first biometric image. The main purpose of this part is to represent the image using high-level semantic features, which not only reduces the dimensionality of the original image space to improve image processing speed, but also facilitates better interaction with subsequent conditional features (i.e., the text feature information of prompt words).

[0268] This application does not limit the specific network structure for image feature encoding and decoding. For example, the image feature encoding and decoding can be a VAE encoder or image feature encoding and decoding information in CLIP.

[0269] In this embodiment of the application, the electronic device not only extracts the first image feature information of the first biometric image, but also extracts the text feature information of the prompt word.

[0270] As shown in Figure 18, this embodiment of the application uses a text encoder to extract text features from prompt words on an electronic device, obtaining the text feature information of the prompt words. This embodiment of the application does not limit the specific network structure of the text encoder; for example, the text encoder can be a CLIP text encoder.

[0271] In addition to using the above-described method to extract the first image feature information of the first biometric image and the text feature information of the prompt word, electronic devices can also use other methods to obtain these features, and this application embodiment does not limit this.

[0272] The specific implementation process of S202 can be referred to the relevant description of S102 above, and will not be repeated here.

[0273] The electronic device extracts the first image feature information of the first biometric image and the text feature information of the prompt word, and performs the following steps S203.

[0274] S203. The electronic device determines the second image feature information corresponding to the preset liveness attack type.

[0275] It should be noted that there is no specific order in the execution of S202 and S203. That is to say, S202 can be executed before S203, after S203, or simultaneously with S203.

[0276] The specific methods for determining the second image feature information corresponding to the preset liveness attack type in this application embodiment include, but are not limited to, the following:

[0277] Method 1: Determine N candidate image feature information corresponding to a preset liveness attack type, and select one candidate image feature information from these N candidate image feature information as the second image feature information, where N is a positive integer. The N candidate image feature information represent the image feature information of N types of liveness attacks belonging to the preset liveness attack type.

[0278] The liveness attack types in this application include 3D masks, paper flipping, screen flipping, 3D head models, and other attack types. In this application, different liveness attack characteristics may be included for different liveness attack types.

[0279] For example, 3D masks and 3D head models differ significantly in material from real living beings, and different materials have different texture characteristics. Therefore, the liveness attack characteristics of 3D masks and 3D head models can include features such as texture.

[0280] For example, compared to real living objects, photographs of paper or screens can exhibit features such as shadows and image edges. Therefore, the characteristics of liveness attacks on photographs of paper or screens can include features such as shadows and image edges.

[0281] In this first method, the electronic device determines N candidate image feature information corresponding to a preset liveness attack type based on the above method. The electronic device selects one candidate image feature information from these N candidate image feature information as the second image feature information. For example, the electronic device may select a default candidate image feature information as the second image feature information. Alternatively, the electronic device may display these N candidate image feature information to an object, allowing the object to select one candidate image feature information from these N candidate image feature information as the second image feature information.

[0282] Method 2: Based on a preset live attack type, acquire a second live attack image and extract the live attack feature information from the second live attack image as the second image feature information.

[0283] For example, if the preset liveness attack type is screen flip, then the second liveness attack image is the image obtained by screen flipping. The electronic device can extract edge feature information, etc., from the second liveness attack image as second image feature information.

[0284] For example, if the preset liveness attack type is a 3D mask or a 3D head model, then the second liveness attack image is an image of a 3D mask or a 3D head model. The electronic device can extract the texture feature information of the second liveness attack image as the second image feature information.

[0285] This application does not limit the specific method by which the electronic device extracts texture feature information from the second liveness attack image. For example, as shown in Figure 18, the electronic device extracts texture feature information from the second liveness attack image through a texture feature extraction module. Optionally, the texture feature extraction module may include gray-level co-occurrence matrix, Tamura texture features, autoregressive texture models, wavelet transform, etc.

[0286] In some embodiments, the aforementioned N candidate image feature information or the live attack feature information of the second live attack image is image texture feature information.

[0287] S204. The electronic device inputs text feature information, first image feature information and second image feature information into the image generation model. The image generation model is configured to generate a first liveness attack image that conforms to the text feature information and second image feature information based on the first image feature information.

[0288] In this embodiment, to ensure that the image generated by the image generation model not only meets the requirements of the prompt word but also possesses image detail features corresponding to the preset live attack type, the image processing first extracts the first image feature information of the first biometric image, extracts the text feature information of the prompt word, and presets the second image feature information corresponding to the live attack type. As shown in Figure 18, the first image feature information of the first biometric image, the text feature information of the prompt word, and the second image feature information are then input into the image generation model. The image generation model processes the first image feature information of the first biometric image using the text feature information of the prompt word and the second image feature information as control conditions to generate the first live attack image. Because the second image feature information and the text feature information of the prompt word are considered when generating the first live attack image, the generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word but also includes the second image feature information corresponding to the preset live attack type, thereby improving the generation effect of the first live attack image.

[0289] The embodiments of this application do not limit the specific network structure of the above image generation model.

[0290] In some embodiments, if the image generation model includes a denoising network, then S204 above includes the following steps S204-A to S204-C:

[0291] S204-A: The electronic device adds noise to the first image feature information of the first biometric image to obtain the noise-added feature information of the first biometric image;

[0292] S204-B: The electronic device uses a denoising network to denoise the noisy feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information, and obtains the denoised image feature information.

[0293] S204-C: The electronic device decodes the denoised image feature information to obtain the first liveness attack image.

[0294] In this implementation, forward diffusion and backward diffusion techniques are used to generate the first liveness attack image. The forward diffusion process adds noise to the image, gradually transforming it into a featureless, noisy image. Backward diffusion is the process of subtracting this estimated noise from the noisy image.

[0295] In this embodiment, the electronic device, based on the above steps, obtains the first image feature information and the second image feature information of the first biometric image, as well as the text feature information of the prompt word, and then performs noise addition processing on the first image feature information of the first biometric image. For example, a preset noise value is added to the first image feature information of the first biometric image to obtain the noisy feature information of the first biometric image. Exemplarily, Gaussian noise can be gradually added to the first image feature information of the first biometric image to obtain the noisy feature information of the first biometric image. This process can be understood as a forward diffusion process.

[0296] Next, the electronic device performs denoising processing on the first biometric image based on the second image feature information and the text feature information of the prompt words, and obtains the denoised image feature information. This process can be understood as a reverse diffusion process.

[0297] In some embodiments, S204-B above includes the following steps S204-B1 to S204-B3:

[0298] S204-B1. The electronic device uses a denoising network to denoise the i-th denoising feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information, and obtains the i-th noise prediction value, where i is a positive integer less than or equal to N. If i is 1, the i-th denoising feature information is the noise-added feature information of the first biometric image.

[0299] S204-B2, The electronic device obtains the (i+1)th denoising feature information of the first biometric image based on the i-th noise prediction value and the i-th denoising feature information of the first biometric image;

[0300] S204-B3: The electronic device takes the (i+1)th denoising feature information as the new ith denoising feature information, returns and repeats the above steps to obtain the Nth denoising feature information of the first biometric image.

[0301] S204-B4 The electronic device determines the Nth denoising feature information as the denoised image feature information.

[0302] In this implementation, the electronic device uses a denoising network to perform multiple denoising processes on the noisy features of the first biometric image, using the textual features of the prompt words and the second image features as control conditions. The denoising network predicts the noise value for each denoising step. Specifically, during denoising, the noise value predicted by the denoising network is subtracted from the previous denoised feature information to generate new denoised feature information, which is then input into the denoising network for further denoising. This process is repeated multiple times until all the noise added to the first biometric image is removed. In the denoising process, in addition to the new denoising feature information of the first biometric image and the text feature information of the prompt, the denoising network also inputs the second image feature information. This makes the image feature information closer to the text feature information of the prompt and the second image feature information during the denoising process. As a result, the image feature information obtained after denoising integrates the text feature information of the prompt and the second image feature information. Thus, the first live attack image generated based on the image feature information obtained after denoising not only meets the requirements of the preset live attack type indicated by the prompt, but also includes the image detail features corresponding to the preset live attack type, thereby improving the image generation effect of the image generation model.

[0303] The following describes the specific process in S204-B1 above, in which the textual feature information of the prompt word and the second image feature information are used as control conditions, and the i-th noise prediction value is obtained by denoising the i-th denoising feature information of the first biometric image through a denoising network.

[0304] In some embodiments, the denoising network includes multiple network layers (e.g., M network layers). During the i-th denoising process, the electronic device uses the text feature information of the prompt word and the second image feature information as control conditions. When the denoising network performs denoising processing on the i-th denoising feature information of the first biometric image, at least one of the text feature information of the prompt word and the second image feature information can be fused with the output feature information of at least some network layers in the denoising network, thereby realizing the binding of the text feature information of the prompt word and the second image feature information as conditions with the distribution of the first biometric image.

[0305] In this embodiment of the application, the electronic device uses the text feature information of the prompt word and the second image feature information as control conditions, and performs denoising processing on the i-th denoising feature information of the first biometric image through a denoising network to obtain the i-th noise prediction value. The specific method is not limited.

[0306] In some embodiments, S204-B1 above includes the following steps S204-B11 and S204-B12:

[0307] S204-B11, The electronic device uses the i-th denoising feature information of the first biometric image as the input feature information of the denoising network;

[0308] S204-B12. The electronic device fuses the feature information output by the first part of the denoising network layer with the text feature information of the prompt word at least, and fuses the feature information output by the second part of the denoising network layer with the second image feature information at least, to obtain the i-th noise prediction value.

[0309] In one example, the denoising network comprises multiple network layers. The electronic device fuses the feature information output by the first part of the denoising network layer with at least the text feature information of the prompt word, and fuses the feature information output by the second part of the denoising network layer with at least the feature information of the second image to obtain the i-th noise prediction value. The first part of the network layer and the second part of the network layer are not entirely identical.

[0310] In one example, the electronic device fuses the feature information output from each layer of the denoising network with the textual feature information of the prompt word and the second image feature information, and uses the fused feature information as the input feature information for the next network layer. This process continues until the feature information output from the last network layer of the denoising network is obtained, which is used as the i-th noise prediction value. The first and second parts of the network layers are identical, both consisting of M network layers in the denoising network.

[0311] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with the text feature information of the prompt word, and fuses the output feature information of the second part of the denoising network layer with the second image feature information. The first and second part of the network layer do not contain any identical network layers. For example, the electronic device fuses feature information 1 output from the first network layer of the denoising network with the text feature information of the prompt word, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is then fused with the second image feature information and used as the input feature information of the third network layer, and so on, to obtain the feature information output from the last network layer of the denoising network, which is used as the i-th noise prediction value.

[0312] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with at least one of the text feature information of the prompt word and the second image feature information, and fuses the output feature information of the second part of the denoising network layer with the text feature information of the prompt word and the second image feature information, wherein the first part of the network layer and the second part of the network layer do not contain the same network layer. For example, the electronic device fuses feature information 1 output by the first network layer of the denoising network with the text feature information of the prompt word, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is fused with the second image feature information and used as the input feature information of the third network layer. The output feature information 3 of the third network layer is fused with the second image feature information and the text feature information of the prompt word and used as the input feature information of the fourth network layer. And so on, to obtain the feature information output by the last network layer of the denoising network, which is used as the i-th noise prediction value.

[0313] This application does not limit the specific method by which the output feature information of some or all layers of the denoising network is fused with the text feature information of the prompt word and at least one of the second image feature information. It only requires ensuring that the denoising network considers both the second image feature information and the text feature information of the prompt word when denoising the i-th denoising feature information of the first biometric image.

[0314] In this embodiment, to facilitate the description of the process of fusing the output feature information of the network layer in the denoising network with the text feature information of the prompt word and at least one of the second image feature information, the k-th network layer in the denoising network is used as an example for explanation. In this case, S204-B12 may include the following steps S204-B12-a and S204-B12-b:

[0315] S204-B12-a. For the kth network layer in the denoising network, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network, where k is a positive integer.

[0316] S204-B12-b: The electronic device obtains the i-th noise prediction value based on the input feature information of the (k+1)-th network layer.

[0317] This application does not limit the specific network structure of the denoising network. For example, it can be any neural network that predicts noise values ​​based on noisy image feature information and text feature information. For example, a U-Net network can be used to remove noise from the image information while simultaneously spreading the denoised image feature information in the direction of the preset liveness attack type indicated by the prompt word and the second image feature information corresponding to the preset liveness attack type, so as to generate second image feature information that satisfies the preset liveness attack type indicated by the prompt word and the second image feature information corresponding to the preset liveness attack type.

[0318] In this embodiment of the application, for the kth network layer of the denoising network, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network.

[0319] This application embodiment does not limit the specific method by which the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network.

[0320] In some embodiments, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information through one or more convolutional layers to obtain the input feature information of the (k+1)th network layer in the denoising network.

[0321] In some embodiments, the electronic device uses an attention mechanism to fuse the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer.

[0322] In one possible implementation, the textual feature information of the prompt word is used as the key value of the attention module, the second image feature information is used as the representative value (V) of the attention module, and the output feature information of the kth network layer is used as the query value (Q) of the attention module. These are then input into the attention module for fusion processing to obtain the input feature information of the (k+1)th network layer.

[0323] Based on the steps described above, the electronic device can determine the feature information output by the last layer of the denoising network. In one example, the electronic device can determine the feature information output by the last layer of the denoising network as the i-th noise prediction value.

[0324] Next, the steps of S204-B2 above are executed to obtain the (i+1)th denoising feature information of the first biometric image based on the i-th noise prediction value and the i-th denoising feature information of the first biometric image. For example, the difference between the i-th denoising feature information and the i-th noise prediction value of the first biometric image is determined as the (i+1)th denoising feature information of the first biometric image.

[0325] The electronic device takes the (i+1)th denoising feature information of the first biometric image determined above as the new ith denoising feature information, returns to execute the above steps S204-B1, repeats the execution N times, obtains the ith denoising feature information of the first biometric image, and then uses the ith denoising feature information as the denoised image feature information.

[0326] Finally, the electronic device performs the steps of S204-C above, that is, the electronic device decodes the denoised image feature information to obtain the first liveness attack image.

[0327] In this embodiment of the application, the denoised image feature information is feature information in the latent space. The electronic device needs to decode the denoised image feature information to convert the denoised image feature information from the latent space to the pixel space to obtain the first liveness attack image.

[0328] In some embodiments, as shown in FIG18, the electronic device decodes the denoised image feature information using a decoder (e.g., a VAE decoder) to obtain a first liveness attack image. This first liveness attack image generates colors based on a prompt word and second image feature information, ensuring that the generated first liveness attack image not only meets the requirements of the preset liveness attack type indicated by the prompt word but also includes the image detail features corresponding to the preset liveness attack type, thereby improving the image generation effect.

[0329] The specific implementation details of S204 can be found in the relevant description of S104 above, and will not be repeated here.

[0330] The image processing method provided in this application embodiment acquires a first biometric image and a prompt word, the prompt word indicating a preset live attack type. Then, image feature extraction is performed on the first biometric image to obtain first image feature information, and text feature extraction is performed on the prompt word to obtain text feature information. Furthermore, second image feature information corresponding to the preset live attack type is determined. The text feature information, the first image feature information, and the second image feature information are then input into an image generation model. Based on the first image feature information, the image generation model generates a first live attack image that conforms to both the text feature information and the second image feature information. Therefore, this application embodiment, when generating a first live attack image based on a first biometric image and a prompt word, uses the second image feature information and the prompt word as control conditions to process the first biometric image and generate the first live attack image. This ensures that the generated first live attack image not only meets the requirements of the preset live attack type indicated by the prompt word but also includes the image features corresponding to that preset live attack type, thereby improving the generation quality of the first live attack image.

[0331] The foregoing provides an overall overview of the image processing method provided in the embodiments of this application. The following describes the process of applying the image processing method of the embodiments of this application to the generation of training samples.

[0332] Figure 19 is a schematic flowchart of a training sample generation method provided in an embodiment of this application. The executing entity of this embodiment can be a device with the function of generating training samples, such as a training sample generation device. This training sample generation device can be the server 102 or terminal device 101 shown in Figure 1, or it can be a system composed of the server 102 or terminal device 101 shown in Figure 1. For ease of description, the following embodiments use an electronic device as an example to illustrate the method of this embodiment.

[0333] As shown in Figure 19, the training sample generation method of this application embodiment includes:

[0334] S301. The electronic device acquires the fifth live attack image and prompt words.

[0335] This prompt indicates the preset liveness attack type.

[0336] The training sample generation method in this application takes the generation of training data for a liveness detection model as an example.

[0337] Liveness detection is often used in conjunction with other technologies, such as facial recognition, in practical applications. Liveness detection serves as the first line of defense and is a crucial step in authentication security. Currently, facial recognition is already being used in various businesses, such as remote bank verification, facial payment, remote driver authentication, and access control systems.

[0338] During the remote account opening process, facial recognition technology and a liveness detection algorithm are used to verify the account holder's identity. The specific process is as follows: First, the user's face is captured by the application's camera at the front end. The front end transmits this image to the back end and calls the algorithm. The algorithm performs a liveness detection and returns the result to the front end. If the user is confirmed to be alive, the verification passes; otherwise, it fails.

[0339] Facial recognition plays a crucial role in facial payment, with liveness detection being a key aspect of payment security. High-precision liveness detection methods can reject illegal attacks that attempt to conduct transactions, ensuring transaction security and protecting the interests of companies and individuals.

[0340] In access control systems, to improve facial recognition efficiency, the system directly acquires facial images at the front end, sends them into a pre-packaged model, performs the judgment, and provides feedback on the results.

[0341] Therefore, the accuracy of the liveness detection model is of paramount importance. Effective training of the liveness detection model is necessary to ensure accurate detection. However, due to the complexity and variety of liveness attack types, relying solely on manual data collection and labeling to construct training samples cannot meet actual business needs. The collected data may be homogeneous and limited, failing to construct effective liveness attack samples, leading to inaccurate detection by the liveness detection model and posing a risk of identity infringement.

[0342] To address this technical problem, this application applies the aforementioned image processing method to training sample generation, proposing a training sample generation method. This method uses a trained image generation model to process the fifth liveness attack image and the prompt word to generate a sixth liveness attack image. Both the fifth and sixth liveness attack images serve as training samples for the liveness detection model. In other words, the method described in this application can amplify the liveness attack samples for the liveness detection model, thereby enabling effective training of the liveness detection model.

[0343] The aforementioned fifth liveness attack image can be a liveness attack image made of a single material. For example, it could be a photograph of a piece of paper, a photograph of a screen, or a 3D head model or 3D mask, etc.

[0344] For example, the prompt indicates a preset liveness attack type. For instance, if the fifth liveness attack image is a 3D mask, the prompt would be "3D Mask, Silica gel," indicating that the material of the 3D mask should be changed to silicone. In other words, the preset liveness attack type indicated by the prompt is a 3D mask made of silicone.

[0345] The specific implementation process of S301 can be referred to the relevant description of S201 above, and will not be repeated here.

[0346] S302. The electronic device performs image feature extraction and texture feature extraction on the fifth live attack image to obtain the image feature information and texture feature information of the fifth live attack image.

[0347] In this embodiment of the application, after the electronic device obtains the fifth live attack image and the prompt word corresponding to the fifth live attack image based on the above steps, in order to achieve the mutual fusion between the image information of the fifth live attack image and the semantic information of the prompt word, the electronic device first extracts the image feature information of the fifth live attack image and the texture feature information of the fifth live attack image.

[0348] In this embodiment of the application, the second image feature information is the texture feature information of the fifth liveness attack image.

[0349] In some embodiments, before extracting the image feature information of the fifth live attack image and before extracting the texture feature information of the fifth live attack image, the electronic device first preprocesses the fifth live attack image to obtain a preprocessed fifth live attack image.

[0350] This application does not limit the specific method of preprocessing the fifth liveness attack image in its embodiments. For example, it may process the clarity, sharpness, etc. of the fifth liveness attack image.

[0351] In one example, the electronic device crops the size of the fifth liveness attack image to match the size of the input image of the image generation model, based on the input image size of the image generation model.

[0352] In some examples, the electronic device performs image matting on the fifth live attack image, extracting the object region from the first biometric image, and then resizing the object region to a first size. The electronic device then crops the first-sized object region to a second size, resulting in the preprocessed fifth live attack image.

[0353] Next, the electronic device performs image feature extraction and texture feature extraction on the preprocessed fifth live attack image.

[0354] As shown in Figure 20, this embodiment of the application encodes the fifth liveness attack image using an image feature encoder to compress the fifth liveness attack image from the pixel space into the latent space, thereby obtaining the image feature information of the fifth liveness attack image. This embodiment of the application does not limit the specific network structure of the image feature encoding and decoding. For example, the image feature encoding and decoding can be a VAE encoder or image feature encoding and decoding information in CLIP.

[0355] This application does not limit the specific method by which the electronic device extracts texture feature information from the fifth liveness attack image. For example, as shown in Figure 20, the electronic device extracts texture feature information from the fifth liveness attack image through a texture feature extraction module. Optionally, the texture feature extraction module may include gray-level co-occurrence matrix, Tamura texture features, autoregressive texture models, wavelet transform, etc.

[0356] The specific implementation process of S302 can be referred to the relevant description of S202 above, and will not be repeated here.

[0357] S303. The electronic device extracts text features from the prompt words to obtain the text feature information of the prompt words.

[0358] It should be noted that there is no specific order in the execution of S302 and S303. That is to say, S302 can be executed before S303, after S303, or simultaneously with S303.

[0359] As shown in Figure 20, this embodiment of the application uses a text encoder to extract text features from prompt words on an electronic device, obtaining the text feature information of the prompt words. This embodiment of the application does not limit the specific network structure of the text encoder; for example, the text encoder can be a CLIP text encoder.

[0360] The specific implementation process of S303 can be referred to the relevant description of S203 above, and will not be repeated here.

[0361] S304. The electronic device inputs the text feature information of the prompt word, the texture feature information of the fifth live attack image, and the image feature information of the fifth live attack image into the image generation model. Based on the image feature information, the image generation model generates a sixth live attack image that conforms to the text feature information and texture feature information.

[0362] As shown in Figure 20, the image feature information of the fifth liveness attack image, the text feature information of the prompt word, and the texture feature information of the fifth liveness attack image are input into the image generation model. The image generation model processes the image feature information of the fifth liveness attack image using the text feature information of the prompt word and the texture feature information of the fifth liveness attack image as control conditions, generating the sixth liveness attack image. Because the generation of the sixth liveness attack image considers the texture feature information of the fifth liveness attack image and the text feature information of the prompt word, the generated sixth liveness attack image not only meets the requirements of the prompt word but also includes the detailed features (such as texture features) of the fifth liveness attack image, thereby improving the generation effect of the sixth liveness attack image.

[0363] The embodiments of this application do not limit the specific network structure of the above image generation model.

[0364] In some embodiments, if the image generation model includes a denoising network, then S304 above includes the following steps S304-A to S304-C:

[0365] S304-A: The electronic device adds noise to the image feature information of the fifth live attack image to obtain the noise-added feature information of the fifth live attack image;

[0366] S304-B: The electronic device uses the textual feature information of the prompt word and the texture feature information of the fifth live attack image as control conditions, and performs denoising processing on the noisy feature information of the fifth live attack image through a denoising network to obtain the denoised image feature information.

[0367] The S304-C electronic device decodes the denoised image feature information to obtain the sixth live attack image.

[0368] In some embodiments, S304-B ​​above includes the following steps S304-B1 to S304-B3:

[0369] S304-B1. The electronic device uses the text feature information of the prompt word and the texture feature information of the fifth live attack image as control conditions. Through a denoising network, it performs denoising processing on the i-th denoising feature information of the fifth live attack image to obtain the i-th noise prediction value. i is a positive integer less than or equal to N. If i is 1, the i-th denoising feature information is the denoised feature information of the fifth live attack image.

[0370] S304-B2, the electronic device obtains the (i+1)th denoising feature information of the fifth live attack image based on the i-th noise prediction value and the i-th denoising feature information of the fifth live attack image;

[0371] S304-B3: The electronic device takes the (i+1)th denoising feature information as the new ith denoising feature information, returns and repeats the above steps to obtain the Nth denoising feature information of the fifth live attack image.

[0372] S304-B4 The electronic device determines the Nth denoising feature information as the denoised image feature information.

[0373] The following describes the specific process of using a denoising network, with the textual features of the prompt words and the texture features of the fifth live attack image as control conditions, to denoise the noisy features of the fifth live attack image and obtain the denoised image features.

[0374] In this embodiment, the denoising process using a denoising network is an iterative process. For example, using the denoising network, with the textual feature information of the prompt words and the texture feature information of the fifth liveness attack image as control conditions, the i-th denoising feature information of the fifth liveness attack image is denoised to obtain the i-th noise prediction value. Then, based on the i-th noise prediction value and the i-th denoising feature information of the fifth liveness attack image, the (i+1)-th denoising feature information of the fifth liveness attack image is obtained. Next, using the (i+1)-th denoising feature information as the new i-th denoising feature information, the denoising network, with the textual feature information of the prompt words and the texture feature information of the fifth liveness attack image as control conditions, denoises the new i-th denoising feature information to obtain the i-th noise prediction value. Based on the i-th noise prediction value and the i-th denoising feature information of the fifth liveness attack image, the (i+1)-th denoising feature information of the fifth liveness attack image is obtained. Next, the denoising feature information of the (i+1)th time is used as the new denoising feature information of the ith time, and the process is repeated. After repeating the denoising operation N times, the denoising feature information of the Nth time can be obtained, and then the denoising feature information of the Nth time is determined as the denoised image feature information.

[0375] The following describes the specific process in S304-B1 above, in which the textual feature information of the prompt word and the texture feature information of the fifth live attack image are used as control conditions. The i-th noise prediction value is obtained by denoising the i-th noise feature information of the fifth live attack image through a denoising network.

[0376] In this embodiment, the denoising network includes multiple network layers (e.g., M network layers). During the i-th denoising process, the electronic device uses the text feature information of the prompt word and the texture feature information of the fifth live attack image as control conditions. When the denoising network performs denoising processing on the i-th denoising feature information of the fifth live attack image, at least one of the text feature information of the prompt word and the texture feature information of the fifth live attack image can be fused with the output feature information of at least some network layers in the denoising network, thereby realizing the binding of the text feature information of the prompt word and the texture feature information of the fifth live attack image as conditions with the distribution of the fifth live attack image.

[0377] In this embodiment, the electronic device uses the textual feature information of the prompt word and the texture feature information of the fifth live attack image as control conditions. The i-th denoising feature information of the fifth live attack image is denoised through a denoising network to obtain the i-th noise prediction value. The specific method is not limited.

[0378] In some embodiments, if the denoising network includes multiple network layers, the above S304-B1 includes the following steps S304-B11 and S304-B12:

[0379] S304-B11, The electronic device uses the i-th denoising feature information of the fifth live attack image as the input feature information of the denoising network;

[0380] S304-B12. The electronic device fuses the feature information output by the first part of the denoising network layer with the text feature information of the prompt word at least, and fuses the feature information output by the second part of the denoising network layer with the texture feature information of the fifth live attack image at least, to obtain the i-th noise prediction value.

[0381] In one example, the denoising network comprises multiple network layers. The electronic device fuses the feature information output by the first part of the denoising network layer with at least the text feature information of the prompt word, and fuses the feature information output by the second part of the denoising network layer with at least the texture feature information of the fifth liveness attack image to obtain the i-th noise prediction value. The first and second part of the network layers are not entirely identical.

[0382] In one example, the electronic device fuses the feature information output from each layer of the denoising network with the textual feature information of the prompt word and the texture feature information of the fifth liveness attack image, and uses the fused feature information as the input feature information for the next network layer. This process continues until the feature information output from the last network layer of the denoising network is obtained, which is used as the i-th noise prediction value. The first and second parts of the network layers are identical, both consisting of M network layers in the denoising network.

[0383] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with the text feature information of the prompt word, and fuses the output feature information of the second part of the denoising network layer with the texture feature information of the fifth liveness attack image. The first and second part of the network layers do not contain the same network layers. For example, the electronic device fuses feature information 1 output from the first network layer of the denoising network with the text feature information of the prompt word, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is then fused with the texture feature information of the fifth liveness attack image, and used as the input feature information of the third network layer, and so on, to obtain the feature information output from the last network layer of the denoising network, which is used as the i-th noise prediction value.

[0384] In one example, the electronic device fuses the output feature information of the first part of the denoising network layer with at least one of the text feature information of the prompt word and the texture feature information of the fifth live attack image. It also fuses the output feature information of the second part of the denoising network layer with the text feature information of the prompt word and the texture feature information of the fifth live attack image. The first and second part of the network layers do not contain the same network layers. For example, the electronic device fuses feature information 1 output from the first network layer of the denoising network with the text feature information of the prompt word, and uses the fused feature information as the input feature information of the second network layer. The output feature information 2 of the second network layer is then fused with the texture feature information of the fifth live attack image, and used as the input feature information of the third network layer. The output feature information 3 of the third network layer is then fused with the texture feature information of the fifth live attack image and the text feature information of the prompt word, and used as the input feature information of the fourth network layer. This process continues until the feature information output by the last network layer of the denoising network is obtained, which is used as the i-th noise prediction value.

[0385] In this embodiment, to facilitate the description of the process of fusing the output feature information of the network layer in the denoising network with at least one of the text feature information of the prompt word and the texture feature information of the fifth liveness attack image, the k-th network layer in the denoising network is used as an example for illustration. In this case, the above S304-B12 may include the following steps S304-B12-a and S304-B12-b:

[0386] S304-B12-a. For the kth network layer in the denoising network, the electronic device fuses the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the texture feature information of the fifth live attack image to obtain the input feature information of the (k+1)th network layer in the denoising network, where k is a positive integer.

[0387] S304-B12-b: The electronic device obtains the i-th noise prediction value based on the input feature information of the (k+1)-th network layer.

[0388] In this embodiment, for the k-th layer of the denoising network, the electronic device fuses the output feature information of the k-th layer with at least one of the text feature information of the prompt word and the texture feature information of the fifth liveness attack image to obtain the input feature information of the (k+1)-th layer of the denoising network. For example, the electronic device fuses the output feature information of the k-th layer with the text feature information of the prompt word and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network. Another example is that the electronic device fuses the output feature information of the k-th layer with the texture feature information of the fifth liveness attack image and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network. Yet another example is that the electronic device fuses the output feature information of the k-th layer with all three of these: the texture feature information of the fifth liveness attack image and the text feature information of the prompt word, and uses the fused feature information as the input feature information of the (k+1)-th layer of the denoising network.

[0389] In some embodiments, the electronic device uses an attention mechanism to fuse the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the texture feature information of the fifth live attack image to obtain the input feature information of the (k+1)th network layer.

[0390] Based on the steps described above, the electronic device can determine the feature information output by the last layer of the denoising network. In one example, the electronic device can determine the feature information output by the last layer of the denoising network as the i-th noise prediction value.

[0391] Next, the steps of S304-B2 above are executed to obtain the (i+1)th denoising feature information of the fifth live attack image based on the i-th noise prediction value and the i-th denoising feature information of the fifth live attack image.

[0392] The electronic device takes the (i+1)th denoising feature information of the fifth live attack image determined above as the new i-th denoising feature information, returns to execute the above steps S304-B1, repeats the execution N times, obtains the N-th denoising feature information of the fifth live attack image, and then uses the N-th denoising feature information as the denoised image feature information.

[0393] Finally, the electronic device performs the steps of S304-C above, that is, the electronic device decodes the denoised image feature information to obtain the sixth liveness attack image.

[0394] In some embodiments, as shown in FIG20, the electronic device decodes the denoised image feature information using a decoder (e.g., a VAE decoder) to obtain a sixth liveness attack image. This sixth liveness attack image is based on the prompt words and the texture feature information of the fifth liveness attack image, transforming the initial features of the object in the fifth liveness attack image into the target features described by the prompt words, thereby improving the image generation effect.

[0395] The specific implementation process of S304 can be referred to the relevant description of S204 above, and will not be repeated here.

[0396] The training sample generation method provided in this application involves acquiring a fifth live attack image and a prompt word, whereby the prompt word instructs the initial features of objects in the fifth live attack image to be transformed into the target features described by the prompt word. Next, image feature extraction and texture feature extraction are performed on the fifth live attack image to obtain its image feature information and texture feature information. Text feature extraction is also performed on the prompt word to obtain its text feature information. Then, using the text feature information of the prompt word and the texture feature information of the fifth live attack image as control conditions, an image generation model processes the image feature information of the fifth live attack image to obtain a sixth live attack image. Based on the fifth and sixth live attack images, the loss of the image generation model is determined, and the image generation model is trained based on this loss. Therefore, it can be seen that when generating a sixth live attack image based on a fifth live attack image and a prompt word in the embodiments of this application, the texture feature information of the fifth live attack image and the prompt word are used as control conditions to process the fifth live attack image and generate a sixth live attack image. The generated sixth live attack image not only includes the features described by the prompt word, but also retains the detailed features of the fifth live attack image, thereby improving the generation quality of the sixth live attack image.

[0397] The method embodiments of this application have been described in detail above with reference to Figures 2 to 20. The apparatus embodiments of this application will be described in detail below with reference to Figure 21.

[0398] Figure 21 is a schematic block diagram of an image processing apparatus provided in an embodiment of this application.

[0399] As shown in Figure 21, the image processing device 10 includes:

[0400] Acquisition unit 11 is used to acquire a first biometric image and a prompt word, wherein the prompt word indicates a preset live attack type;

[0401] The feature extraction unit 12 is used to extract the first image feature information of the first biometric image and the text feature information of the prompt word;

[0402] Determining unit 13 is used to determine the second image feature information corresponding to the preset liveness attack type;

[0403] Processing unit 14 is used to input the text feature information, the first image feature information and the second image feature information into an image generation model, wherein the image generation model is configured to generate a first liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information.

[0404] In some embodiments, the determining unit 13 is specifically used to determine N candidate image feature information corresponding to the preset liveness attack type, wherein the N candidate image feature information are image feature information of N kinds of liveness attacks belonging to the preset liveness attack type, and N is a positive integer.

[0405] In some embodiments, the N candidate image feature information or the live attack feature information of the second live attack image is image texture feature information.

[0406] In some embodiments, the determining unit 13 is specifically used to obtain a second live attack image based on the preset live attack type; and to extract the live attack feature information of the second live attack image as the second image feature information.

[0407] In some embodiments, the image generation model includes a denoising network and a processing unit 14, specifically configured to: add noise to the first image feature information of the first biometric image to obtain denoised feature information of the first biometric image; denoise the i-th denoised feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information through the denoising network to obtain the i-th noise prediction value, where i is a positive integer less than or equal to N; if i is 1, the i-th denoised feature information is the denoised feature information of the first biometric image; obtain the (i+1)-th denoised feature information of the first biometric image based on the i-th noise prediction value and the i-th denoised feature information of the first biometric image; use the (i+1)-th denoised feature information as the new i-th denoised feature information, and repeat the above steps to obtain the N-th denoised feature information of the first biometric image; determine the N-th denoised feature information as the denoised image feature information; and decode the denoised image feature information to obtain the first live attack image.

[0408] In some embodiments, the processing unit 14 is specifically used to take the i-th denoising feature information of the first biometric image as the input feature information of the denoising network; to fuse the feature information output by the first part of the network layer of the denoising network with at least the text feature information of the prompt word; and to fuse the feature information output by the second part of the network layer of the denoising network with at least the second image feature information to obtain the i-th noise prediction value.

[0409] In some embodiments, the processing unit 14 is specifically configured to, for the kth network layer in the denoising network, fuse the output feature information of the kth network layer with at least one of the text feature information of the prompt word and the second image feature information to obtain the input feature information of the (k+1)th network layer in the denoising network, where k is a positive integer; and obtain the i-th noise prediction value based on the input feature information of the (k+1)th network layer.

[0410] In some embodiments, the processing unit 14 is specifically used to take the text feature information of the prompt word as the key value of the attention module, take the second image feature information as the representative value of the attention module, take the output feature information of the kth network layer as the query value of the attention module, and input them into the attention module for fusion processing to obtain the input feature information of the (k+1)th network layer.

[0411] In some embodiments, the feature extraction unit 12 is specifically used to preprocess the first biometric image to obtain a preprocessed first biometric image; and to extract image features from the preprocessed first biometric image to obtain first image feature information of the first biometric image.

[0412] In some embodiments, the feature extraction unit 12 is specifically used to perform image cutout processing on the first biometric image to cut out the object region in the first biometric image; adjust the size of the object region to a first size; and crop the object region of the first size to a second size to obtain the preprocessed first biometric image.

[0413] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details are omitted here. Specifically, the device shown in FIG21 can perform the above-described image processing method embodiments, and the foregoing and other operations and / or functions of each module in the device are respectively for implementing the above-described method embodiments, which will not be repeated here for the sake of brevity.

[0414] Figure 22 is a schematic block diagram of a model training device provided in an embodiment of this application.

[0415] As shown in Figure 22, the model training device 20 includes:

[0416] Acquisition unit 21 is used to acquire training images and prompt words, wherein the prompt words indicate a preset live attack type, and the training images are biometric images or live attack images;

[0417] Feature extraction unit 22 is used to extract the first image feature information of the training image and the text feature information of the prompt word;

[0418] Determining unit 23 is used to determine the second image feature information corresponding to the preset liveness attack type;

[0419] Processing unit 24 is used to input the text feature information of the prompt word, the first image feature information and the second image feature information into an image generation model, wherein the image generation model is configured to generate a third liveness attack image that conforms to the text feature information and the second image feature information based on the first image feature information;

[0420] Training unit 25 is used to determine the loss of the image generation model based on the third liveness attack image and the feature information of the second image, and to train the image generation model based on the loss.

[0421] In some embodiments, the determining unit 23 is specifically used to determine N candidate image feature information corresponding to the preset liveness attack type, wherein the N candidate image feature information are image feature information of N kinds of liveness attacks belonging to the preset liveness attack type, and N is a positive integer; and select one candidate image feature information from the N candidate image feature information as the second image feature information.

[0422] In some embodiments, the determining unit 23 is specifically used to extract the live attack feature information of the training image as the second image feature information if the training image is a live attack image.

[0423] In some embodiments, the determining unit 23 is specifically used to obtain a fourth live attack image based on the preset live attack type, and extract the live attack feature information of the fourth live attack image as the second image feature information.

[0424] In some embodiments, the training unit 25 is specifically used to extract live attack features from the third live attack image to obtain live attack feature information of the third live attack image; and to determine the loss of the image generation model based on the second image feature information and the live attack feature information of the third live attack image.

[0425] In some embodiments, if the second image feature information is the texture feature information of the training image, then the training unit 25 is specifically used to extract texture features from the third live attack image to obtain the texture feature information of the third live attack image; and to determine the loss of the image generation model based on the texture feature information of the training image and the texture feature information of the third live attack image.

[0426] In some embodiments, the training unit 25 specifically determines the difference between the texture feature information of the training image and the texture feature information of the third liveness attack image; and determines the loss of the image generation model based on the difference.

[0427] In some embodiments, the feature extraction unit 22 is specifically used to preprocess the training image to obtain a preprocessed training image; and to extract image features from the preprocessed training image to obtain the first image feature information of the training image.

[0428] In some embodiments, the feature extraction unit 22 is specifically used to perform image matting on the training image to extract object regions in the training image; adjust the size of the object regions to a first size; and randomly crop the object regions of the first size to a second size to obtain the preprocessed training image.

[0429] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, they will not be repeated here. Specifically, the device shown in FIG22 can execute the above-described model training method embodiments, and the foregoing and other operations and / or functions of each module in the device are respectively for implementing the above-described method embodiments, which will not be repeated here for the sake of brevity.

[0430] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0431] Figure 23 is a schematic block diagram of an electronic device provided in an embodiment of this application. The electronic device may be the terminal device or server described above.

[0432] As shown in Figure 23, the electronic device 40 may include:

[0433] The system includes a memory 41 and a processor 42. The memory 41 stores a computer program 43 and transfers the computer program 43 to the processor 42. In other words, the processor 42 can retrieve and run the computer program 43 from the memory 41 to implement the methods described in the embodiments of this application.

[0434] For example, the processor 42 can be used to execute the steps in the above method according to the instructions in the computer program 43.

[0435] In some embodiments of this application, the processor 42 may include, but is not limited to:

[0436] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0437] In some embodiments of this application, the memory 41 includes, but is not limited to:

[0438] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0439] In some embodiments of this application, the computer program 43 may be divided into one or more modules, which are stored in the memory 41 and executed by the processor 42 to perform the method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 43 in the electronic device.

[0440] As shown in Figure 23, the electronic device 40 may further include:

[0441] Transceiver 44, which can be connected to processor 42 or memory 41.

[0442] The processor 42 can control the transceiver 44 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 44 may include a transmitter and a receiver. The transceiver 44 may further include antennas, and the number of antennas may be one or more.

[0443] It should be understood that the various components in the electronic device 40 are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0444] According to one aspect of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0445] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above-described method embodiments.

[0446] In other words, when implemented using software, it can be implemented wholly or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0447] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0448] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0449] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0450] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized by, The method comprises: obtaining a first biometric image and a prompt word, the prompt word indicating a preset live body attack type; extracting first image feature information of the first biometric image and text feature information of the prompt word; determining second image feature information corresponding to the preset live body attack type; inputting the text feature information, the first image feature information and the second image feature information into an image generation model, the image generation model being configured to generate a first live body attack image conforming to the text feature information and the second image feature information based on the first image feature information.

2. The method of claim 1, wherein, The determination of the second image feature information corresponding to the preset live body attack type comprises: determining N candidate image feature information corresponding to the preset live body attack type, the N candidate image feature information being image feature information of N live body attacks belonging to the preset live body attack type, N being a positive integer; selecting one candidate parameter feature information from the N candidate image feature information as the second image feature information.

3. The method of claim 1, wherein, The determination of the second image feature information corresponding to the preset live body attack type comprises: obtaining a second live body attack image based on the preset live body attack type; extracting live body attack feature information of the second live body attack image as the second image feature information.

4. The method according to claim 2 or 3, characterized in that, The image feature information of the N candidate image feature information or the second live body attack image is image texture feature information.

5. The method according to any one of claims 1 to 4, characterized in that, The image generation model comprises a denoising network, and the inputting of the text feature information, the first image feature information and the second image feature information into the image generation model, the image generation model being configured to generate a first live body attack image conforming to the text feature information and the second image feature information based on the first image feature information, comprises: performing noise adding processing on the first image feature information to obtain noise-added feature information of the first biometric image; performing denoising processing on the i-th denoised feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information through the denoising network to obtain an i-th noise prediction value, i being a positive integer less than or equal to N, if i is 1, the i-th denoised feature information being the noise-added feature information of the first biometric image; obtaining i+1-th denoised feature information of the first biometric image based on the i-th noise prediction value and the i-th denoised feature information of the first biometric image; taking the i+1-th denoised feature information as new i-th denoised feature information and returning to repeat the execution to obtain N-th denoised feature information of the first biometric image; determining the N-th denoised feature information as denoised image feature information; performing decoding processing on the denoised image feature information to obtain the first live body attack image.

6. The method of claim 5, wherein, The denoising processing on the i-th denoised feature information of the first biometric image based on the text feature information of the prompt word and the second image feature information through the denoising network to obtain an i-th noise prediction value comprises: The i-th denoising feature information of the first biometric feature image is taken as input feature information of the denoising network; The feature information output by the first part of the network layer of the denoising network is fused with at least the text feature information of the prompt word, and the feature information output by the second part of the network layer of the denoising network is fused with at least the second image feature information, to obtain the i-th noise prediction value.

7. The method of claim 6, wherein, The feature information output by the first part of the network layer of the denoising network is fused with at least the text feature information of the prompt word, and the feature information output by the second part of the network layer of the denoising network is fused with at least the second image feature information, to obtain the i-th noise prediction value, including: For the k-th network layer in the denoising network, the output feature information of the k-th network layer is fused with at least one of the text feature information of the prompt word and the second image feature information, to obtain input feature information of the k+1-th network layer in the denoising network, where k is a positive integer; Based on the input feature information of the k+1-th network layer, the i-th noise prediction value is obtained.

8. The method of claim 7, wherein, The feature information output by the first part of the network layer of the denoising network is fused with at least the text feature information of the prompt word, and the feature information output by the second part of the network layer of the denoising network is fused with at least the second image feature information, to obtain the i-th noise prediction value, including: The text feature information of the prompt word is taken as a key value of an attention module, the second image feature information is taken as a representative value of the attention module, and the output feature information of the k-th network layer is taken as a query value of the attention module, which are input into the attention module for fusion processing to obtain the input feature information of the k+1-th network layer.

9. The method according to any one of claims 1 to 8, characterized in that, Before the first image feature information of the first biometric feature image is extracted, the method further includes: Performing matting processing on the first biometric feature image to cut out an object region in the first biometric feature image; Adjusting the size of the object region to a first size; Center cropping the object region of the first size to a second size to obtain a preprocessed first biometric feature image; The first image feature information of the first biometric feature image is extracted, including: Performing image feature extraction on the preprocessed first biometric feature image to obtain the first image feature information of the first biometric feature image.

10. A model training method, comprising: Including: Obtaining a training image and a prompt word, the prompt word indicating a preset live body attack type, and the training image being a biometric feature image or a live body attack image; Extracting first image feature information of the training image and text feature information of the prompt word; Determining second image feature information corresponding to the preset live body attack type; Inputting the text feature information of the prompt word, the first image feature information, and the second image feature information into an image generation model, the image generation model being configured to generate a third live body attack image conforming to the text feature information and the second image feature information based on the first image feature information; Determine a loss of the image generation model based on the third live attack image and the second image feature information, and train the image generation model based on the loss.

11. The method of claim 10, wherein, The determining of the second image feature information corresponding to the preset live attack type comprises: Determine N candidate image feature information corresponding to the preset live attack type, wherein the N candidate image feature information is image feature information of N live attacks belonging to the preset live attack type, and N is a positive integer. Select one candidate image feature information from the N candidate image feature information as the second image feature information.

12. The method of claim 10, wherein, The determining of the second image feature information corresponding to the preset live attack type comprises: If the training image is a live attack image, extract live attack feature information of the training image as the second image feature information.

13. The method of claim 10, wherein, The determining of the second image feature information corresponding to the preset live attack type comprises: Based on the preset live attack type, obtain a fourth live attack image; Extract live attack feature information of the fourth live attack image as the second image feature information.

14. The method according to any one of claims 10 to 13, characterized in that, The determining of the loss of the image generation model based on the third live attack image and the second image feature information comprises: Perform live attack feature extraction on the third live attack image to obtain live attack feature information of the third live attack image; Determine the loss of the image generation model based on the second image feature information and the live attack feature information of the third live attack image.

15. The method of claim 14, wherein, If the second image feature information is texture feature information of the training image, the live attack feature extraction on the third live attack image to obtain the live attack feature information of the third live attack image comprises: Perform texture feature extraction on the third live attack image to obtain texture feature information of the third live attack image. The determining of the loss of the image generation model based on the second image feature information and the live attack feature information of the third live attack image comprises: Determine the loss of the image generation model based on the texture feature information of the training image and the texture feature information of the third live attack image.

16. The method of claim 15, wherein, The determining of the loss of the image generation model based on the texture feature information of the training image and the texture feature information of the third live attack image comprises: Determine the difference between the texture feature information of the training image and the texture feature information of the third live attack image; Determine the loss of the image generation model based on the difference.

17. The method according to any one of claims 10-16, characterized in that, Before the extracting of the first image feature information of the training image, the method further comprises: Perform matting processing on the training image to cut out an object region in the training image; Adjust the size of the object region to a first size; Randomly crop the object region of the first size to a second size to obtain a preprocessed training image; The extracting of the first image feature information of the training image comprises: Image feature extraction is performed on the preprocessed training image to obtain first image feature information of the training image.

18. An image processing apparatus characterized by comprising: The method comprises the following steps: An acquisition unit is configured to acquire a first biometric image and a prompt word, wherein the prompt word indicates a preset liveness attack type. A feature extraction unit is configured to extract first image feature information of the first biometric image and text feature information of the prompt word. A determination unit is configured to determine second image feature information corresponding to the preset liveness attack type. A processing unit is configured to input the text feature information, the first image feature information, and the second image feature information into an image generation model, wherein the image generation model is configured to generate a first liveness attack image that meets the text feature information and the second image feature information based on the first image feature information.

19. A model training apparatus, comprising: The method comprises the following steps: An acquisition unit is configured to acquire a training image and a prompt word, wherein the prompt word indicates a preset liveness attack type, and the training image is a biometric image or a liveness attack image. A feature extraction unit is configured to extract first image feature information of the training image and text feature information of the prompt word. A determination unit is configured to determine second image feature information corresponding to the preset liveness attack type. A processing unit is configured to input the text feature information of the prompt word, the first image feature information, and the second image feature information into an image generation model, wherein the image generation model is configured to generate a third liveness attack image that meets the text feature information and the second image feature information based on the first image feature information. A training unit is configured to determine a loss of the image generation model based on the second image feature information and the third liveness attack image, and train the image generation model based on the loss.

20. An electronic device, comprising: The device comprises a processor and a memory; The memory is configured to store a computer program; The processor is configured to execute the computer program to implement the method according to any one of claims 1 to 9 or 10 to 17.

21. A computer-readable storage medium, characterized in that, The computer program is configured to enable a computer to execute the method according to any one of claims 1 to 9 or 10 to 17. The computer program comprises computer instructions configured to enable a processor to implement the method according to any one of claims 1 to 9 or 10 to 17.

22. A computer program product, characterised in that, ​

Images