Image generation method, model training method, device and electronic equipment

Abstract

The present application relates to the field of computer technology, and in particular to an image generation method, a model training method, a device and an electronic device. The image generation method comprises: acquiring sampling features, performing feature mapping on the sampling features by a driving feature generation model to obtain a first identity feature that obeys a first distribution, wherein the first distribution is a feature distribution obeyed by the identity features extracted by an object recognition model, and then driving the image generation model to perform diffusion denoising on a plurality of different noise images respectively according to the first identity features, so as to obtain a plurality of different object images of identity identification represented by the first identity features; since the first identity feature and the identity feature extracted by the object recognition model obey the same feature distribution, the feature distribution consistency between the training process of the object recognition model and the generation process of the object image is ensured, thereby improving the ability of the object image to maintain the identity feature, and facilitating different object images of the same identity to maintain the same identity feature.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an image generation method, a model training method, a device and an electronic device. Background Art

[0002] In the field of artificial intelligence technology, identity recognition through images is an important development direction, such as face recognition, fingerprint recognition, iris recognition, etc. The neural network model used for identity recognition is called an object recognition model. In related technologies, in order to avoid infringement or leakage of privacy, images automatically generated by image generation models are used to replace real collected images to train object recognition models.

[0003] In the related art, there is a problem that images generated by image generation models cannot maintain identity consistency. Identity consistency means that different images generated for the same identity identifier are essentially inconsistent in identity. Since different images generated for the same identity identifier cannot maintain identity consistency, this leads to poor training results for the object recognition model. Summary of the invention

[0004] In view of this, the embodiments of the present application propose an image generation method, a model training method, an apparatus and an electronic device. The method maps the sampled features to obtain a first identity feature that obeys a first distribution. Since the first identity feature and the identity feature extracted by the object recognition model obey the same feature distribution, it can be ensured that multiple object images generated by driving the first identity feature have identity consistency, thereby ensuring the training effect of the object recognition model.

[0005] The embodiment of the present application is implemented by adopting the following technical solutions:

[0006] In a first aspect, an embodiment of the present application provides an image generation method, comprising: obtaining sampling features; performing feature mapping on the sampling features by a driving feature generation model to obtain a first identity feature that obeys a first distribution, wherein the first distribution is the feature distribution obeyed by the identity feature extracted by an object recognition model; using the first identity feature, driving the image generation model to perform diffusion denoising on a plurality of different noise images respectively to obtain a plurality of different object images represented by the first identity feature, wherein the plurality of different object images represented by the first identity feature are used to train the object recognition model.

[0007] In second aspect, an embodiment of the present application provides a method for training an object recognition model, the method comprising: obtaining training data, the training data comprising multiple different object images of the same identity identification, the multiple different object images of the same identity identification being generated according to the above image generation method; iteratively training the object recognition model according to the multiple different object images of the same identity identification until a fourth training end condition is reached.

[0008] In a third aspect, an embodiment of the present application provides an image generating device, comprising: a first acquisition module, for acquiring sampling features; a mapping module, for driving a feature generation model to perform feature mapping on the sampling features to obtain a first identity feature that obeys a first distribution, wherein the first distribution is a feature distribution obeyed by the identity feature extracted by an object recognition model; an image generation module, for driving an image generation model to perform diffusion denoising on a plurality of different noise images respectively through the first identity feature to obtain a plurality of different object images represented by the first identity feature, wherein the plurality of different object images represented by the first identity feature are used to train the object recognition model.

[0009] In some embodiments, the image generation model includes a diffusion network and a decoder network; the image generation module includes a denoising unit and a decoding unit, and for each of the noise images in a plurality of different noise images, the denoising unit is used to use the first identity feature as a diffusion condition by the diffusion network to perform T rounds of diffusion denoising on the noise image to obtain a target denoising feature output by the T round of diffusion denoising; wherein the denoising feature output by the i-th round of diffusion denoising is used as an input for the i+1-th round of diffusion denoising, i and T are positive integers, i<T; the decoding unit is used to use the decoder network to decode the target denoising feature to obtain an object image represented by the identity feature.

[0010] In some embodiments, the image generation device further includes a first training preparation module, which is used to obtain a first sample object image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image, wherein the first sample identity feature is obtained by extracting features from the first sample object image using a reference object recognition model, and the reference object recognition model is the object recognition model or another model with the same recognition task as the object recognition model; k is a positive integer not exceeding T; a first encoding module, which is used to encode the first sample object image using an encoder network to obtain a first encoding feature; a first denoising module, which is used to perform k rounds of diffusion denoising on the first encoding feature using the diffusion network to obtain a diffusion denoising result; the diffusion denoising result includes a first The target sample denoising feature is obtained by k rounds of diffusion denoising; the first denoising module is used to perform k rounds of diffusion denoising on the target sample denoising feature with the first sample identity feature as a diffusion condition by the diffusion network to obtain the sample denoising feature; the first decoding module is used to decode the sample denoising feature by the decoder network to obtain the object decoded image; the first extraction module is used to extract the identity feature of the object decoded image by the reference object recognition model to obtain the reference identity feature corresponding to the sample denoising feature; the first loss calculation module is used to determine the identity feature constraint loss according to the first sample identity feature and the reference identity feature; the first parameter adjustment module is used to adjust at least the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

[0011] In some embodiments, the first loss calculation module includes a compensation coefficient determination unit, which is used to determine the compensation coefficient for the reference round k, and the compensation coefficient is negatively correlated with the reference round k; a first calculation unit, which is used to calculate the feature loss between the first sample identity feature and the reference identity feature; and a compensation unit, which is used to compensate for the feature loss by the compensation coefficient to obtain the identity feature constraint loss.

[0012] In some embodiments, the diffusion denoising result also includes the noise predicted in the k-round diffusion denoising process, and the image generating device also includes a diffusion loss calculation module, which is used to determine the diffusion loss according to the noise added by the k-round diffusion denoising of the first coding feature and the noise predicted in the process of performing k-round diffusion denoising on the target sample noisy feature; the first parameter adjustment module also includes a weighted calculation unit, which is used to weight the identity feature constraint loss and the diffusion loss to obtain a first loss; and an adjustment unit, which is used to adjust at least the parameters of the diffusion network according to the first loss until the first training end condition is reached.

[0013] In some embodiments, the first parameter adjustment module is further used to fix the parameters of the encoder network and the decoder network, and adjust the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

[0014] In some embodiments, the image generating device further includes a second training preparation module for acquiring a second sample object image; a second encoding module for performing feature extraction on the second sample object image by the encoder network to obtain a second encoded feature; a second decoding module for decoding the second encoded feature by the decoder network to obtain a decoded sample object image; a second loss calculation module for determining a second loss based on the second sample object image and the decoded sample object image; and a second parameter adjustment module for adjusting the parameters of the encoder network and the decoder network based on the second loss until a second training end condition is reached.

[0015] In some implementations, the first acquisition module is used to sample in the coding feature distribution to obtain the sampled feature; the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding.

[0016] In some embodiments, the image generation device also includes a third training preparation module, which is used to obtain a third sample object image and a third sample identity feature corresponding to the third sample object image, wherein the third sample identity feature is obtained by extracting features from the third sample object image by the reference object recognition model; a second mapping module, which is used to map the third sample identity feature to the coding feature distribution by a distribution mapping network to obtain predicted distribution parameters; the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding; a feature restoration module, which is used to perform feature restoration based on the predicted distribution parameters by the driving feature generation model to obtain sample driving features that obey the first distribution; a third loss calculation module, which is used to determine a third loss based on the sample driving feature and the third sample identity feature; and a third parameter adjustment module, which is used to adjust the parameters of the driving feature generation model and the distribution mapping network according to the third loss until the third training end condition is reached.

[0017] In some implementations, the object recognition model is a face recognition model, and the object image is a face image.

[0018] In a fourth aspect, an embodiment of the present application provides a training device for an object recognition model, the device comprising: a second acquisition module, used to acquire training data, the training data comprising multiple different object images for each identity identification, the multiple different object images for each identity identification being generated according to the above-mentioned image generation method; an object recognition model training module, used to iteratively train the object recognition model according to the multiple different object images for each identity identification until a fourth training termination condition is reached.

[0019] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor; a memory, wherein computer instructions are stored in the memory, and when the computer instructions are executed by the processor, the above method is implemented.

[0020] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the above-mentioned method is implemented.

[0021] In a seventh aspect, an embodiment of the present application provides a computer program product, including computer instructions, which implement the above method when executed by a processor.

[0022] An image generation method, device and electronic device provided by the embodiment of the present application, after acquiring the sampling feature, the driving feature generation model first performs feature mapping on the sampling feature to obtain a first identity feature that obeys a first distribution; through the first identity feature, the driving image generation model performs diffusion denoising on multiple different noise images respectively to obtain multiple different object images of the identity represented by the first identity feature. Since the first distribution is the feature distribution obeyed by the identity feature extracted by the object recognition model, the first identity feature obeys the first distribution, and the distribution obeyed by the first identity feature is consistent with the feature distribution obeyed by the identity feature extracted by the object recognition model, that is, the sampling feature is mapped to the feature space where the feature extracted by the object recognition model is located, and the first identity feature used to drive the generation of the object image is kept consistent with the feature distribution of the training object recognition model process. In this way, in the process of image generation, compared with the sampling feature, the first identity feature has a stronger ability to maintain identity information and a stronger ability to control identity information in the image generation process. Therefore, it can be ensured that different object images generated under the drive of the same first identity feature correspond to the same identity, which effectively solves the problem that the image generated by the image generation model in the related art cannot maintain identity consistency.

[0023] These and other aspects of the present application will become more clearly understood in the description of the following embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0025] Figure 1 A schematic diagram of an application scenario provided in an embodiment of the present application is shown.

[0026] Figure 2 A schematic flow chart of an image generation method provided in an embodiment of the present application is shown.

[0027] Figure 3 It shows that an embodiment of the present application provides Figure 2 Schematic diagram of the process of step S130.

[0028] Figure 4A A schematic diagram of image generation by an image generation model shown in an embodiment of the present application is shown.

[0029] Figure 4B A schematic diagram of image generation by an image generation model shown in another embodiment of the present application is shown.

[0030] Figure 5A A second flow chart of an image generating method provided in an embodiment of the present application is shown.

[0031] Figure 5B A schematic diagram of processing a diffusion network during a training process according to an embodiment of the present application is shown.

[0032] Figure 6 It shows that an embodiment of the present application provides Figure 5A Schematic diagram of the process of step S270.

[0033] Figure 7 A third flow chart of the image generating method provided in an embodiment of the present application is shown.

[0034] Figure 8 A fourth flow chart of the image generating method provided in an embodiment of the present application is shown.

[0035] Fig. 9 A training diagram of an object recognition model provided in an embodiment of the present application is shown.

[0036] Fig.10 A schematic diagram of the training process of the encoder network and the decoder network provided in an embodiment of the present application is shown.

[0037] Fig.11A schematic diagram of the training process of a diffusion network provided in an embodiment of the present application is shown.

[0038] Fig.12 A schematic diagram of the training process of a driving feature generation model provided in an embodiment of the present application is shown.

[0039] Fig.13 An application flow chart of an image generation method according to an embodiment of the present application is shown.

[0040] Fig.14 A schematic diagram of an image generating device provided in an embodiment of the present application is shown.

[0041] Fig.15 A schematic diagram of a training device for an object recognition model provided in an embodiment of the present application is shown.

[0042] Fig.16 A schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0043] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0044] In order to enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present application.

[0045] In the following description, the terms "first\second" and the like are merely used to distinguish similar objects and do not represent a specific ordering of the objects. It is understandable that "first\second" can be interchanged with a specific order or sequence where permitted, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0046] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0047] Before further describing the embodiments of the present application in detail, the nouns and terms involved in the embodiments of the present application are explained. The nouns and terms involved in the embodiments of the present application are subject to the following interpretations.

[0048] Diffusion network: It is a neural network used to automatically generate images. The training process of diffusion network involves the process of adding noise and denoising. The process of adding noise (also called forward process, forward diffusion process) gradually adds noise to the input data (the input data can be the image features of the image). In the denoising process (also called reverse process, reverse reconstruction process), the noise is predicted and gradually denoised to restore the image features without noise or with less noise, and then restore the image through the restored image features. Diffusion network is essentially a Markov architecture that can provide powerful image generation capabilities.

[0049] Encoder network: In this application, it refers to a neural network used to digitally encode images. The encoder network can also compress the input high-dimensional image into low-dimensional coding features, reducing the dimension of the data and reducing the subsequent calculation amount.

[0050] Decoder network: In this application, it refers to the neural network used to restore image features to images.

[0051] In the field of artificial intelligence technology, identity recognition through images is an important development direction, such as face recognition, fingerprint recognition, iris recognition, etc. The neural network model used for identity recognition is called an object recognition model. In related technologies, in order to avoid infringement or leakage of privacy, images automatically generated by image generation models are used to replace real collected images to train object recognition models.

[0052] In the related art, there is a problem that images generated by image generation models cannot maintain identity consistency. Identity consistency means that different images generated for the same identity identifier are essentially inconsistent in identity. Since different images generated for the same identity identifier cannot maintain identity consistency, this leads to poor training results for the object recognition model.

[0053] In order to solve the above problems, the present application provides an image generation method, a model training method, an apparatus and an electronic device, wherein, after acquiring the sampling features, the image generation method drives the feature generation model to perform feature mapping on the sampling features to obtain a first identity feature that obeys a first distribution, wherein the first distribution is the feature distribution obeyed by the identity features extracted by the object recognition model; through the first identity feature, the image generation model is driven to perform diffusion denoising on multiple different noise images respectively to obtain multiple different object images of the identity represented by the first identity feature, that is, different object images corresponding to the same identity.

[0054] Through the method provided in the embodiment of the present application, since the first distribution is the feature distribution obeyed by the identity feature extracted by the object recognition model, the first identity feature obeys the first distribution, and the distribution obeyed by the first identity feature is consistent with the feature distribution obeyed by the identity feature extracted by the object recognition model, that is, the sampling feature is mapped to the feature space where the feature extracted by the object recognition model is located, and the first identity feature used to drive the generation of the object image is kept consistent with the feature distribution of the training object recognition model process. In this way, during the image generation process, compared with the sampling feature, the first identity feature has a stronger ability to retain identity information and a stronger ability to control identity information in the image generation process. Therefore, it can be ensured that different object images generated under the drive of the same first identity feature correspond to the same identity identification, which effectively solves the problem that the image generated by the image generation model in the related technology cannot maintain identity consistency.

[0055] See also Figure 1 , Figure 1 It is a schematic diagram of an application scenario involved in an embodiment of the present application, including an electronic device 10, which may be a server, a terminal or other device with computing capabilities. The electronic device 10 may be used to execute the method of the present application, and the electronic device 10 is connected to a database 20 via a network, wherein the network may be a wide area network or a local area network, or a combination of the two.

[0056] A variety of neural network models can be deployed in the electronic device 10, such as Figure 1 In the driving feature generation model and the image generation model, when the electronic device 10 executes the method provided in the present application, the electronic device 10 can obtain a noise image from the database 20, and the driving feature generation model deployed in the electronic device 10 performs feature mapping on the sampled features to obtain a first identity feature that obeys a first distribution, and the first distribution is the feature distribution obeyed by the identity feature extracted by the object recognition model; through the first identity feature, the image generation model deployed in the driving electronic device 10 is driven to perform diffusion denoising on multiple different noise images respectively, and obtain multiple different object images represented by the first identity feature.

[0057] The electronic device 10 may also deploy an object recognition model ( Figure 1 (not shown) after obtaining a plurality of different object images of the identity represented by the first identity feature, the electronic device 10 can train an object recognition model through the plurality of different object images.

[0058] Among them, the terminal can be a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart TV, a wearable device, a smart watch, a virtual reality device, a car terminal, a smart TV, etc., but is not limited to these.

[0059] The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, as well as big data and artificial intelligence platforms.

[0060] The method provided in the embodiment of the present application will be described below in conjunction with exemplary applications and implementations of the electronic device provided in the embodiment of the present application.

[0061] See also Figure 2 , Figure 2 A flow chart of an image generation method provided in an embodiment of the present application is given. The method can be executed by an electronic device, such as a server, etc. The image generation method includes steps S110-S130:

[0062] S110: Obtain sampling features.

[0063] In some embodiments, the sampling feature may refer to a feature obtained by sampling from the image coding feature space. Since the image coding feature space can be described by the distribution obeyed by the features in the image coding feature space, and the image coding feature space is the feature space where the features obtained by the encoder network encoding are located, the distribution obeyed by the features in the image coding feature space is also the distribution obeyed by the coding features obtained by the encoder network encoding. The distribution obeyed by the coding features obtained by the encoder network encoding is called the coding feature distribution, and therefore, the sampling feature can also be understood as a feature sampled from the coding feature distribution. It is worth mentioning that in the present application, the encoder network refers to a neural network used to encode an image, and the corresponding coding features obtained by the encoder network encoding are also image features.

[0064] In some embodiments, random sampling may be performed to obtain sampling features. In other embodiments, since it is necessary to generate multiple object images for training the object recognition model, feature sampling may be performed in a preset order to obtain sampling features for each sampling. In some embodiments, the coding feature distribution may be a Gaussian distribution. In other embodiments, the coding feature distribution may also be a Laplace distribution, a Cauchy distribution, a gamma distribution, etc., which are not specifically limited here.

[0065] The image coding feature space is learned by the encoder network based on the object images in the data set, or in other words, the image coding feature space is a feature space formed by the image features extracted by the encoder network from the object images in the data set. The image coding feature space is a multidimensional space, and each object image in the data set can be mapped to a point in the image coding feature space. The point mapped by the object image in the image coding feature space is determined by the image features of the image (i.e., the image features extracted by the encoder network from the object image). The object images in the data set can be used to train the encoder network to determine the image coding feature space.

[0066] The object images in the data set are related to the recognition task of the object recognition model. If the recognition task of the object recognition model is a face recognition task, then the object recognition model is a face recognition model, and the object images in the data set are face images; if the recognition task of the object recognition model is an iris recognition task, then the object recognition model is an iris recognition model, and the object images in the data set are iris images; if the recognition task of the object recognition model is a fingerprint recognition task, then the object recognition model is a fingerprint recognition model, and the object images in the data set are fingerprint images; if the recognition task of the object recognition model is a palmprint recognition task, then the object recognition model is a palmprint recognition model, and the object images in the data set are palmprint images.

[0067] In some embodiments, the sampling features may be image features extracted from the reference object image by the trained encoder network, that is, the reference object image is mapped to the image coding feature space by the trained encoder network to obtain the image features of the reference object image. The sampling features are used as the basis for image generation to generate multiple different object images corresponding to the same identity identifier as the reference object image. Similarly, the reference object image here is also related to the recognition task of the object recognition model. The data set is pre-established; the data set includes object images corresponding to different identity identifiers.

[0068] It should be noted that in the data set of this application, the object images and reference object images involved, when the above embodiments of this application are applied to specific products or technologies, the relevant data collection, use and processing processes should comply with the requirements of national laws and regulations. Before collecting image data, the information processing rules should be informed and the separate consent of the target object should be sought. The image data should be processed in strict accordance with the requirements of laws and regulations and the personal information processing rules, and technical measures should be taken to ensure the security of relevant data.

[0069] S120. The driving feature generation model performs feature mapping on the sampled features to obtain a first identity feature that obeys a first distribution. The first distribution is a feature distribution obeyed by the identity feature extracted by the object recognition model.

[0070] The first identity feature refers to converting the sampled feature into a feature that obeys the first distribution. In the present application, the driving feature generation model refers to a neural network model used to convert the sampled feature into a feature that obeys the first distribution. The driving feature generation model can learn the first distribution during the training process, and then learn the ability to convert the input feature into a feature that obeys the first distribution. The driving feature generation model can be a neural network model constructed by a convolutional neural network, a fully connected network, a pooling neural network, etc. The training process of the driving feature generation model will be specifically described in subsequent embodiments.

[0071] In the present application, the first identity feature is used to drive the image generation model to generate the object image in the subsequent process. Therefore, the first identity feature can also be called the driving feature of the image generation model.

[0072] The object recognition model is a neural network model constructed by one or more neural networks, such as convolutional neural networks, fully connected neural networks, pooling neural networks, etc., which are not specifically limited here. The object recognition model is used to recognize the identity represented by the object image through the object image, or in other words, the object recognition model recognizes the identity of the living object from which the object image comes through the object image. The living object from which the object image comes refers to the living object from which the image content presented in the object image comes, such as humans, animals, plants, etc.

[0073] For a person, face, iris, fingerprint, palm print, etc. can uniquely identify a person. Therefore, the person from whom the object image comes can be uniquely identified through object images such as face image, iris image, fingerprint image, palm print image, etc. Uniquely identifying the person from whom the object image comes through the object image is to identify the identity represented by the object image through the object image. One identity is used to represent a person, and different identity identifiers are used to represent different people.

[0074] As described above, the recognition task of the object recognition model is to recognize the identity of the living object from which the object image comes through the object image. Then, the features extracted from the object image by the object recognition model during the recognition process are more concerned with features related to the identity of the living object. Therefore, in this application, the features extracted by the object recognition model for the input image are called identity features.

[0075] In some embodiments, the identity features extracted by the object recognition model can be introduced during the training of the driving feature generation model to supervise the training process of the driving feature generation model, so that the driving feature generation model learns the first distribution obeyed by the identity features extracted by the object recognition model during the training process. In this way, after training, even if the relevant information of the first distribution (such as the distribution parameters of the first distribution) is not input to the driving feature generation model, the driving feature generation model can convert the sampled features into the first identity features obeying the first distribution.

[0076] In other embodiments, object image generation and object recognition model training may be performed alternately, and one alternating round includes an object recognition model training phase and an object image generation phase. In the object recognition model training phase, training data (the training data includes multiple sample object images and identity identifiers annotated for the sample object images, and in the first alternating round, the sample object images in the training data may be obtained from an existing data set or may be object images collected for living objects) are used to train the object recognition model, and in the process of training the object recognition model, identity features extracted by the object recognition model for the sample object images are output, and distribution parameters of the first distribution are determined based on the identity features extracted by the object recognition model for the multiple sample object images, for example, if the first distribution is a Gaussian distribution, its distribution parameters include a mean and a variance.

[0077] In the object image generation stage, the distribution parameters of the first distribution can be used as the input of the driving feature generation model. In this case, the input of the driving feature generation model includes the distribution parameters of the first distribution and the sampling features, so that the driving feature generation model transforms the sampling features into features that obey the first distribution according to the distribution parameters of the first distribution, obtains the first identity features, and drives the generation of object images according to the subsequent process, and adds the generated object images to the first set. The object images in the first set can be used as training data in the training stage of the object recognition model in the next alternating round to continue training the object recognition model and obtain more accurate distribution parameters of the first distribution, and in the object image generation stage in the next alternating round, the distribution parameters of the first distribution determined in the next alternating round are used as the input of the driving feature generation model in the next alternating round, and the above process is repeated.

[0078] As can be seen from the above, in different alternating rounds, the parameters of the first distribution of the input driving feature generation model may be different. Of course, in different alternating rounds, the sampling features of the input driving feature generation model may also be different. It can be understood that as the alternating rounds increase, the recognition accuracy of the object recognition model becomes higher and higher, and correspondingly, the accuracy of the identity features extracted by the object recognition model becomes higher and higher. Therefore, the distribution parameters of the first distribution obtained in the later alternating rounds are more accurate than the distribution parameters of the first distribution obtained in the previous alternating rounds.

[0079] In step S120, feature mapping refers to mapping the sampled features into a feature space that obeys a first distribution; in the present application, the sampled features are mapped into first identity features that obey the first distribution.

[0080] S130. Drive the image generation model to perform diffusion denoising on a plurality of different noise images respectively through the first identity feature to obtain a plurality of different object images of the identity represented by the first identity feature. The plurality of different object images of the identity represented by the first identity feature are used to train the object recognition model.

[0081] An image generation model refers to a neural network model with the function of automatically generating images. In the present application, the image generation model at least includes a diffusion network, which uses the diffusion denoising capability of the diffusion network to recover noise-free features from a noisy image, and performs image decoding based on the features output by the diffusion network to obtain an object image.

[0082] In some embodiments, the image generation model may include a diffusion network and a decoder network, wherein the diffusion network is used to perform diffusion denoising, and the decoder network is used to perform image decoding on the denoised features output by the diffusion network to output an object image.

[0083] In step S130, the first identity feature can be used as a diffusion condition of the diffusion network, so as to supervise the diffusion network through the first identity feature in the process of diffusion denoising to ensure that the denoised features obtained by each diffusion denoising can maintain the first identity feature, and then ensure that the subsequently generated object image can maintain the first identity feature, that is, the image content presented in the generated object image is the image of the object identified by the first identity feature.

[0084] Driven by the same first identity feature, by inputting different noise images into the image generation model, multiple different object images corresponding to the identity represented by the first identity feature can be obtained, that is, different object images generated by the image generation model driven by the same first identity feature can maintain stable identity information (that is, the first identity feature).

[0085] See also Figure 3 , Figure 3 The embodiment of the present application provides Figure 2 The process diagram of step S130 in FIG. 1 is a flow diagram of step S130, wherein the image generation model includes a diffusion network and a decoder network. In step S130, each noise image can be processed according to the process shown in steps S131-S132:

[0086] S131. The diffusion network uses the first identity feature as a diffusion condition to perform T rounds of diffusion denoising on the noisy image to obtain the target denoising feature output by the T-th round of diffusion denoising; wherein the denoising feature output by the i-th round of diffusion denoising is used as the input of the i+1-th round of diffusion denoising, i and T are positive integers, and i<T.

[0087] It should be noted that it is necessary to first extract features from the noise image to obtain the image features of the noise image. For ease of description, the image features of the noise image are referred to as noise features. The noise features are then input into the diffusion network for the first round of diffusion denoising. T can be set according to actual needs, for example, T is 800, 900, 1000, etc., and T is the total number of denoising rounds in one image generation process.

[0088] The first denoising feature refers to the denoising feature of the output of the Tth round of diffusion denoising in T rounds of diffusion denoising. In each round of diffusion denoising (taking the kth round of diffusion denoising as an example, k is a positive integer, k≤T), the diffusion network performs noise prediction based on the input features of the kth round of diffusion denoising and the first identity feature to obtain the predicted noise of the kth round of diffusion denoising; then, the predicted noise of the kth round of diffusion denoising is removed from the input features of the kth round of diffusion denoising to obtain the denoising feature of the output of the kth round of diffusion denoising. Among them, when k=1, the input feature of the kth round of diffusion denoising is the noise feature; when k is greater than 1, the input feature of the kth round of diffusion denoising is the denoising feature of the output of the k-1th round of diffusion denoising.

[0089] In some embodiments, T rounds of diffusion denoising are performed on a noisy image, and the noise of each round of diffusion denoising can be predicted first, and then the noise predicted for the T rounds of diffusion denoising can be denoised at one time to obtain a target denoising feature.

[0090] In some embodiments, the diffusion network may have a residual connected U-net, which includes a multi-layer convolutional network. In each round of diffusion denoising, the first identity feature may be fused with the input feature of the kth round of diffusion denoising based on an attention mechanism to accurately predict noise. For example, the dot product attention may be calculated based on the attention mechanism.

[0091] S132. The decoder network decodes the target denoising feature to obtain an object image represented by the first identity feature.

[0092] Figure 4A FIG. 1 is a schematic diagram of image generation according to an image generation model shown in an embodiment of the present application. Figure 4A As shown, after the noise features are extracted based on the noise image, the noise features and the first identity features are input into the diffusion network for the first round of diffusion processing. In the first round of diffusion processing, the first identity features can be fused with the features extracted by other network layers (such as convolutional network layers) in the diffusion network based on the attention mechanism, for example, according to Figure 4A The dot product attention is calculated in the manner shown in Q (query), k (key), and V (Value) to predict the predicted noise of the first round of diffusion denoising; then, the predicted noise of the first round of diffusion denoising is removed from the input features of the first round of diffusion denoising to obtain the denoised features output by the first round of diffusion denoising; then, the first identity features and the denoised features output by the first round of diffusion denoising are subjected to the first round of diffusion denoising in a similar manner as described above until the Tth round of diffusion denoising is completed to obtain the first denoised features output by the Tth round of diffusion denoising. Afterwards, the first denoised features are input into the decoder network for image decoding to obtain the object image.

[0093] When the same first identity feature is input, different object images that retain the first identity feature can be obtained by changing the input noise image according to the above process.

[0094] In other embodiments, an environment description text may be introduced to constrain the image generation process, wherein the environment description text is used to limit the environment in the object image, such as the lighting environment, the light and dark environment, the object placement environment (such as the front, the side), and the environment description text is such as a face image under light, a face image in a cool indoor environment, etc. The environment description text may be subjected to semantic feature extraction from the environment description text through a text feature encoder to obtain text semantic features of the environment description text, and the text semantic features and the first identity features may be used as diffusion conditions to perform T rounds of diffusion denoising on the noise image to obtain the first denoising feature output by the T rounds of diffusion denoising; thereafter, the first denoising feature may be decoded through a decoder network to obtain an object image represented by the identity feature.

[0095] Figure 4B is a schematic diagram of image generation according to an image generation model shown in another embodiment of the present application. Figure 4A In the embodiment of Figure 4B In the diffusion network, the diffusion condition includes not only the first identity feature, but also the text semantic feature. Figure 4B The processing process of each round of diffusion denoising is similar to Figure 4AThe corresponding embodiments are similar, except that, in the case of attention fusion, both the first identity feature and the text semantic feature participate in attention fusion.

[0096] While keeping the first identity feature unchanged, by changing the environment description text, object images that maintain the first identity feature and present different environments can be obtained. In this way, the object images presenting different environments obtained can simulate object images in different environments in actual scenes. In this way, object images that maintain the first identity feature and present different environments are used to train the object recognition model, which can improve the recognition accuracy of the object recognition model for images in different environments, so that the object recognition model can be adapted to perform object recognition on images collected in a variety of environments.

[0097] In the related art, the method of generating an object image is: inputting sampling features into an image generation model to generate an object image. The inventor of the present application has found through research that the sampling features are obtained by sampling in the image coding feature space, and the distribution obeyed by the sampling features is inconsistent with the feature distribution during the training of the object recognition model. As a result, in the process of driving the image generation model by the same sampling feature to generate an image, the sampling feature has low control over the identity information maintained during the image generation process, which in turn leads to the problem that the image generated by the image generation model cannot maintain identity consistency, that is, different object images generated by the same sampling feature do not actually correspond to the same identity identifier.

[0098] According to the method provided by the embodiment of the present application, after the sampling feature is obtained, the driving feature generation model first performs feature mapping on the sampling feature to obtain the first identity feature that obeys the first distribution; through the first identity feature, the image generation model is driven to perform diffusion denoising on multiple different noise images respectively to obtain multiple different object images of the identity represented by the first identity feature. Since the first distribution is the feature distribution obeyed by the identity feature extracted by the object recognition model, the first identity feature obeys the first distribution, and the distribution obeyed by the first identity feature is consistent with the feature distribution obeyed by the identity feature extracted by the object recognition model, that is, the sampling feature is mapped to the feature space where the feature extracted by the object recognition model is located, and the first identity feature used to drive the generation of the object image is kept consistent with the feature distribution of the training object recognition model process. In this way, in the process of image generation, compared with the sampling feature, the first identity feature has a stronger ability to maintain identity information and a stronger ability to control identity information in the image generation process. Therefore, it can be ensured that different object images generated under the drive of the same first identity feature correspond to the same identity, which effectively solves the problem that the image generated by the image generation model in the related art cannot maintain identity consistency.

[0099] In addition, in the related art, when using a diffusion network to generate an object image, image diversity is introduced through random masks; that is, after determining the sampling features, the diffusion network is driven to generate different images of the same identity by randomly masking the sampling features. However, since the features are randomly masked, different data noises may be introduced in model training, which causes the quality of the generated object image to deteriorate. At the same time, since the sampling features after random masking are not consistent with the original sampling features, the sampling features after random masking may not represent the same identity.

[0100] The method provided in the embodiment of the present application uses a first identity feature to drive an image generation model to perform diffusion denoising on multiple different noise images respectively; in the process of generating multiple object images, the first identity feature used as a driver always remains consistent, thereby solving the problem in the related art that the noise introduced by random masking and the sampling features after random masking cannot represent the same identity identification.

[0101] See also Figure 5A , Figure 5A A second flow chart of the image generation method involved in the embodiment of the application is given. Before step S130, the image generation method further includes steps S210-S280:

[0102] S210. Obtain a first sample object image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image, wherein the first sample identity feature is obtained by performing feature extraction on the first sample object image with reference to an object recognition model; k is a positive integer not exceeding T.

[0103] The reference object recognition model may be the object recognition model involved in the aforementioned embodiment, or another model having the same recognition task as the object recognition model involved in the aforementioned embodiment.

[0104] As described in the above embodiment, the recognition task of the object recognition model is to recognize the identity of the living object from which the object image comes through the object image; for different neural network models with the same recognition task, the features extracted from the object image are also identity features that can characterize the identity, and for different neural network models with the same recognition task, the distribution differences obeyed by the extracted identity features are small, so the identity features extracted from the object image by different neural network models with the same recognition task can be approximately considered to obey the same distribution. Based on this, the first sample object image can be feature extracted by other models with the same recognition task as the object recognition model to obtain the first sample identity feature.

[0105] In some embodiments, when the reference object recognition model is another model that performs the same recognition task as the object recognition model, the reference object recognition model may be a trained neural network model to ensure the accuracy of the extracted identity features and thereby ensure the training effect of the image generation model.

[0106] As described above, although the distribution differences of the extracted identity features of different neural network models with the same recognition task are small, the reference object recognition model can be the object recognition model to be trained. In this way, it is ensured that the feature distributions obeyed by the first sample identity feature and the third sample identity feature below are both the first distribution, which can avoid the poor training effect of the image generation model caused by the identity features (the first sample identity features and the third sample identity features below) used as supervisory signals during the training process being inconsistent with the actual distribution of the features extracted by the object recognition model. Therefore, the first sample identity features extracted from the first sample object image by the object recognition model to be trained are used as supervisory signals in the training process of the image generation model, which can ensure the training effect of the image generation model.

[0107] The sample object image used to train the diffusion network in the image generation model is called the first sample image, and the identity feature obtained by characterizing the first sample object image with the reference object recognition model is called the first identity feature. As above, the first sample object image is also related to the recognition task of the object recognition model, that is, the image content presented in the first sample image is the recognition target corresponding to the recognition task of the object recognition model. For example, if the recognition target is a face, the first sample object image is a face image. In some embodiments, the first sample object image can be obtained from the data set mentioned above.

[0108] In some embodiments, feature extraction can be performed on the object images in the data set in advance through the reference object recognition model to obtain the identity features of each object image in the data set, and the identity features of the object images are associated with the object images and stored in the data set. In this way, during the training of the diffusion network, there is no need to call the reference object recognition model to extract features from the first sample object image, but the first sample identity features corresponding to the first sample object image can be read from the data set, which can shorten the training time of the diffusion network and improve the training efficiency.

[0109] S220: The encoder network performs encoding processing on the first sample object image to obtain a first encoding feature.

[0110] The first encoding feature refers to an image feature obtained by encoding the first sample object image by the encoder network. The encoder network may be a convolutional neural network, which may involve operations such as convolution calculation, nonlinear activation function (ReLU) calculation, and pooling calculation.

[0111] In some implementations, before step S220, the first sample object image may be scaled to a specified size, such as scaling the first sample object image to 256*256. Image scaling may remove redundant information in the image and reduce the size of the image, thereby reducing the amount of data that needs to be processed during encoding, thereby improving encoding efficiency.

[0112] In some implementations, on the basis of performing image scaling on the first sample object image, the scaled image may be further normalized to facilitate subsequent data calculations.

[0113] S230, performing k rounds of diffusion noise addition on the first coding feature by the diffusion network to obtain a diffusion noise addition result; the diffusion noise addition result includes the target sample noise addition feature obtained by the k-th round of diffusion noise addition.

[0114] Among them, diffusion noise is to add noise to the first coding feature; it can be understood that the noise feature obtained by the nth round of diffusion noise is used as the input of the n+1th round of diffusion noise, n is a positive integer, n≤k. The target sample noise feature refers to the noise feature output by the kth round of diffusion noise in the kth round of diffusion noise on the first coding feature.

[0115] In some embodiments, the noise added in different rounds of diffusion noise addition in the k-round diffusion noise addition of the first coding feature may be noise sampled from the same noise distribution, for example, may be randomly sampled. The noise distribution may be a Gaussian distribution.

[0116] S240, using the diffusion network to perform k rounds of diffusion denoising on the target sample noisy feature using the first sample identity feature as a diffusion condition to obtain a sample denoising feature.

[0117] Different numbers of denoising rounds may be specified for different first sample object images. In some embodiments, the number of denoising rounds may be randomly specified for the first sample object images. In some embodiments, when the diffusion network is trained in batches, the same number of denoising rounds may be specified for the first sample object images of the same batch.

[0118] In a specific embodiment, a designated number of denoising rounds may be embedded for the first sample object image to obtain corresponding embedded features, and the obtained embedded features may be used as inputs of a diffusion network to guide the diffusion network to perform diffusion denoising rounds on the target sample denoising features.

[0119] The output of each round of diffusion denoising can be calculated using the following formula (1):

[0120] ; Calculation formula (1);

[0121] Among them, the output of the diffusion denoising module Used to describe that under the diffusion condition of the first identity feature y, the noise predicted in the tth round of diffusion denoising is noise The probability of represents the diffusion network, are the parameters of the diffusion network, represents the output of the diffusion network in the tth round of diffusion denoising, ,in, is the noise coefficient corresponding to the i-th round of diffusion noise. The value range of is 0-1 and can be set as needed. y represents the first identity feature.

[0122] The sample denoising feature is the denoising feature obtained by performing k rounds of diffusion denoising on the target sample denoising feature, that is, the denoising feature output by performing the k-th round of diffusion denoising on the target sample denoising feature. The diffusion network uses the first sample identity feature as the diffusion condition, and the process of performing k rounds of diffusion denoising on the target sample denoising feature can be referred to the above description, which will not be repeated here.

[0123] S250, the decoder network decodes the sample denoising features to obtain an object decoded image.

[0124] Decoding is the process of restoring and decoding the sample denoising features to the pixel space (i.e., RGB space). The image obtained by decoding the sample denoising features by the decoder network is called the object decoded image. The decoder network is constructed through a convolutional neural network, which may involve operations such as convolution calculation, nonlinear activation function (ReLU) calculation, and pooling calculation.

[0125] S260 , extracting identity features from the object decoded image using a reference object recognition model to obtain reference identity features corresponding to the sample denoising features.

[0126] The reference identity feature corresponding to the sample denoising feature refers to the identity feature obtained by extracting features from the object decoded image using a reference object recognition model.

[0127] It can be understood that the more accurate the noise predicted by the diffusion network in each round of diffusion denoising during the diffusion denoising process, the higher the degree of restoration of the object decoded image compared to the first sample object image, that is, the smaller the difference between the object decoded image and the first sample object image, and correspondingly, the smaller the difference between the reference identity feature extracted from the object decoded image by the reference object recognition model and the first sample identity feature corresponding to the first sample object image.

[0128] It can be understood that the larger the selected k is, the more rounds of diffusion denoising are performed, and relatively speaking, the denoising is more thorough. Correspondingly, the smaller the difference between the object decoded image and the first sample object image is, and the smaller the difference between the reference identity feature extracted by the reference object recognition model from the object decoded image and the first sample identity feature of the first sample object image is.

[0129] Figure 5B is a schematic diagram of processing a diffusion network during training according to an embodiment of the present application, such as Figure 5B As shown, the encoder network encodes the first sample object image, obtains the first encoding feature, and then diffuses and noises the first encoding feature to obtain the target sample noisy feature; then, using the first sample identity feature as a diffusion condition, the target sample noisy feature is diffused and denoised to obtain the sample denoised feature; and the sample denoised feature is input into the decoder network, and the decoder network decodes and outputs the object decoded image.

[0130] S270. Determine the identity feature constraint loss according to the first sample identity feature and the reference identity feature.

[0131] The identity feature constraint loss can be cosine similarity loss, contrast loss, cross entropy loss, etc.; the specific loss calculation method can be selected according to actual needs.

[0132] The identity feature constraint loss is used to reflect the difference between the first sample identity feature and the reference identity feature. It can be understood that the purpose of training is to narrow the difference between the first sample identity feature and the reference identity feature, that is, the smaller the difference between the two, the higher the similarity between the object decoded image and the first sample object image, and the more likely the object decoded image and the first sample object image correspond to the same identity.

[0133] In some embodiments, the difference between the first sample identity feature and the reference identity feature can be calculated by a loss function to obtain the identity feature constraint loss. The loss function is, for example, an absolute value loss function, a mean square error loss function, a cosine similarity loss function, and the like.

[0134] In some embodiments, based on the cosine similarity loss function, the cosine distance between the first sample identity feature and the reference identity feature can be calculated, and the identity feature constraint loss can be determined according to the cosine distance, wherein the identity feature constraint loss is negatively correlated with the cosine distance. For example, the difference between 1 and the cosine distance can be used as the identity feature constraint loss.

[0135] In some embodiments, see Figure 6 , Figure 6 The application example provides Figure 5A Schematic diagram of the process of step S270, step S270 may include steps S271-S273:

[0136] S271. Determine a compensation coefficient according to the reference round k, where the compensation coefficient is negatively correlated with the reference round k.

[0137] In some embodiments, the diffusion denoising round corresponding to the sample denoising feature may be divided by the maximum diffusion denoising round to obtain a diffusion denoising ratio, and then a compensation coefficient may be determined based on the diffusion denoising ratio. For example, the difference between 1 and the diffusion denoising ratio may be used as the compensation coefficient.

[0138] In some embodiments, the compensation factor It can be calculated by the following formula (2):

[0139] ; Calculation formula (2)

[0140] Wherein, t is the reference round determined for the first sample object image, T is the preset maximum diffusion denoising round, exp() represents the natural exponential function, that is, the exponential function with the real number e as the base, and a is the proportional coefficient, which is greater than 0 and less than 1. a can be set according to actual needs, for example, a is 0.5.

[0141] S272. Calculate the feature loss between the first sample identity feature and the reference identity feature.

[0142] In a specific embodiment, the feature loss between the first sample identity feature and the reference identity feature may be calculated using a loss function such as an absolute value loss function, a mean square error loss function, or a cosine similarity loss function.

[0143] S273. Compensate the feature loss through the compensation coefficient to obtain the identity feature constraint loss.

[0144] In some embodiments, the compensation amount may be determined based on the compensation coefficient, and then the compensation amount is added to the feature loss to obtain the identity feature constraint loss. For example, the compensation coefficient may be multiplied by a reference compensation amount to obtain the compensation amount, and the reference compensation amount is a non-negative number.

[0145] In some implementations, taking the identity feature constraint loss as the cosine similarity loss as an example, the compensation coefficient may be applied to the cosine similarity between the first sample identity feature and the reference identity feature. The identity feature constraint loss may be obtained by the following calculation formula (3): for:

[0146] ; Calculation formula (3);

[0147] in, is the compensation coefficient, is the first sample identity feature, is the object recognition model, Decode the image for the object, For reference identity characteristics, " is the modulus length operator.

[0148] That is, in the above calculation formula (3), the cosine similarity between the first sample identity feature and the reference identity feature is calculated, the compensation coefficient is multiplied by the cosine similarity to obtain the multiplication result, and the difference between 1 and the multiplication result is used as the identity feature constraint loss.

[0149] In some embodiments, the first sample identity feature and reference identity characteristics are all normalized feature vectors, and their modulus lengths are all 1. On this basis, formula (3) can be simplified to the following formula (4), identity feature constraint loss for:

[0150] ; Calculation formula (4).

[0151] From the above description, it can be seen that the larger the round of diffusion denoising, the closer the object decoded image is to the first sample object image, and the smaller the gap between the reference identity feature and the first sample identity feature. If the feature loss between the first sample identity feature and the reference identity feature is directly used as the identity feature constraint loss, the influence of the diffusion denoising round will be ignored. Therefore, in this application, the feature loss between the first sample identity feature and the reference identity feature is compensated by the diffusion coefficient that is negatively correlated with the reference round k to obtain the identity feature constraint loss. In this way, the identity feature constraint loss can comprehensively reflect the feature loss between the first sample identity feature and the reference identity feature, as well as the influence of the reference round k on the identity feature constraint loss.

[0152] However, the present application adds a compensation coefficient based on the reference identity feature and the first sample identity feature. Since the compensation coefficient is negatively correlated with the diffusion denoising round (i.e., the reference round k), the smaller the diffusion denoising round, the larger the compensation coefficient, and the larger the diffusion denoising round, the smaller the compensation coefficient. Therefore, the compensation coefficient can be used to effectively balance the impact of the diffusion denoising round on the identity feature constraint loss.

[0153] Taking the above calculation formula (4) as an example, when the diffusion denoising round t is large, the compensation coefficient Smaller, cosine similarity Larger; when the diffusion denoising round t is small, the compensation coefficient Larger, cosine similarity Obviously, whether it is a larger diffusion denoising round or a smaller diffusion denoising round, a stable identity feature constraint loss can be obtained. .

[0154] S280, adjusting at least the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

[0155] In some embodiments, the parameters of the network can be adjusted based on the gradient descent method according to the identity feature constraint loss, wherein the network whose parameters are adjusted includes at least a diffusion network. In some embodiments, the network whose parameters are adjusted based on the identity feature constraint loss can include a diffusion network, an encoder network, and a decoder network.

[0156] In some embodiments, step S280 includes: fixing the parameters of the encoder network and the decoder network, and adjusting the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached. In this case, the encoder network and the decoder network can be trained before training the diffusion network, so that the trained encoder network and the decoder network can be used to assist in training the diffusion network according to the above process.

[0157] The first training end condition may be that the model loss is less than a preset first loss threshold, or that the number of iterations of the diffusion network reaches a first set threshold. The specific first training end condition may be selected according to actual needs. The model loss is determined based on the identity feature constraint loss. For example, the identity feature constraint loss may be used as the model loss.

[0158] In the above training process, the identity features extracted by the reference object recognition model (including the first identity features extracted from the first sample object image, and the reference identity features extracted) are used to assist in training the diffusion network. This can ensure that after training, the diffusion network can, driven by the identity features, recover features that can maintain the identity features from the noise image, and then decode the object image represented by the identity features.

[0159] The identity feature constraint loss constrains the difference between the reference identity feature corresponding to the object decoded image and the first sample identity feature corresponding to the first sample object image. The parameters of the diffusion network are adjusted by minimizing the identity feature constraint loss so that the object image generated by the trained image generation model corresponds to the identity represented by the input identity feature.

[0160] In some embodiments, the diffusion denoising result also includes the noise predicted in k rounds of diffusion denoising processing. Based on steps S210-S270, the image generation method may also include: determining the diffusion loss based on the noise predicted in the process of performing k rounds of diffusion denoising processing on the first coding feature and k rounds of diffusion denoising processing on the target sample noisy feature.

[0161] The noise added in the process of performing diffusion noising on the first coding feature includes all noise added from the first round of diffusion noising to the Kth round of diffusion noising. The noise predicted in the process of performing K rounds of diffusion denoising on the target sample noisy feature includes all noise predicted for the first round of diffusion denoising to the Kth round of diffusion denoising.

[0162] The noise predicted during the k-round diffusion denoising process refers to all the noise removed from the 1st round to the kth round.

[0163] In some embodiments, the diffusion loss may be a mean square error (MSE), and the diffusion loss may be calculated by the following formula (5): for:

[0164] ; Calculation formula (5)

[0165] in, is the sample denoising feature, is the noise added during the diffusion noise addition process. To diffuse the network, is the parameter of the diffusion network, t is the reference round determined for the first sample object image (i.e., k in the above text), ranging from 0 to 1000, , The noise coefficient for the i-th round of diffusion denoising is between 0 and 1, y is the first sample identity feature of the input diffusion model, and E represents the expected operation.

[0166] 2) Weight the identity feature constraint loss and the diffusion loss to obtain the first loss;

[0167] According to the first loss, at least the parameters of the diffusion network are adjusted until a first training end condition is reached.

[0168] Specifically, the first loss for:

[0169] ; Calculation formula (6).

[0170] Wherein, c and d are weighting coefficients, and both c and d are greater than 0. For example, c and d can take the value of 1.

[0171] As above, only the parameters of the diffusion network may be adjusted according to the first loss, that is, the parameters of the encoder network and the decoder network are fixed, and the parameters of the diffusion network are adjusted according to the identity feature constraint loss until the first training end condition is reached.

[0172] In other implementations, the parameters of the encoder network, the decoder network, and the diffusion network may be adjusted according to the first loss.

[0173] In some implementations, based on encoding the first sample object image using the encoder network to obtain the first encoding feature, step S110 may specifically include:

[0174] Sampling is performed in the encoding feature distribution to obtain sampling features; the encoding feature distribution refers to the distribution obeyed by the encoding features obtained by the encoder network encoding.

[0175] See also Figure 7 , Figure 7 A third flow chart of the image generation method provided in an embodiment of the present application is given. Before step S220, the image generation method further includes steps S310-S3500. Steps S310-S350 are the process of training the encoder network and the decoder network:

[0176] S310: Acquire a second sample object image.

[0177] The object image used to train the encoder network and the decoder network is referred to as a second sample image. The second sample object image may be the same as or different from the first sample object image, which is not specifically limited herein. The second sample object image may also be obtained from the data set described above.

[0178] S320. The encoder network performs feature extraction on the second sample object image to obtain a second encoding feature.

[0179] The encoder network extracts features from the second sample object image and compresses the second sample object image from the RGB space to a high-dimensional space, which not only greatly reduces the subsequent calculation dimension, but also the output second encoded features still retain the semantic and representation information of the second sample object image.

[0180] S330. The decoder network decodes the second encoded feature to obtain a decoded sample object image.

[0181] The decoder network decodes the encoded features, that is, restores and decodes the second encoded features into the RGB space, thereby obtaining a decoded sample object image.

[0182] S340: Determine a second loss according to the second sample object image and the decoded sample object image.

[0183] Among them, the second sample object image and the decoded sample object image can be subjected to loss calculation using a target loss function to obtain a second loss. The target loss function can be a cross entropy loss, a mean square error loss function, an absolute value loss function, or the like.

[0184] S350, adjusting parameters of the encoder network and the decoder network according to the second loss until a second training end condition is reached.

[0185] Specifically, according to the second loss, based on the gradient descent method (such as stochastic gradient descent, stochastic gradient descent with momentum term), adjust the parameters of the encoder network and the decoder network to minimize the second loss; repeat the above steps S310-S350 using the adjusted encoder network and decoder network until the second training end condition is reached.

[0186] The second training end condition may be that the second loss is less than a second threshold, or that the number of training iterations of the encoder network and the decoder network reaches a set second number threshold.

[0187] Through the method provided in the embodiment of the present application, the second sample image is encoded through an encoder network to obtain encoding features, and the processing dimension of the second sample image is reduced through the encoding features, thereby improving the efficiency of subsequent second sample image processing; at the same time, the encoding features are decoded through a decoder network to obtain a decoded object image, and the loss between the decoded object image and the second sample image is minimized so that the decoded object image is consistent with the second sample image, thereby ensuring the accuracy of feature encoding by the trained encoder network and the accuracy of image decoding by the trained decoder network.

[0188] In some implementations, VAE (Varitional Auto-Encoder) constraints may be added to the output position of the encoder network to perform constraint training on the encoder network and the decoder network. Specifically, the second encoding feature output by the encoder network may be input into the distribution prediction network to obtain predicted distribution parameters (such as mean and variance); then, the divergence loss between the distribution indicated by the predicted distribution parameters and the first specified distribution (such as Gaussian distribution) is calculated.

[0189] On this basis, the divergence loss and the second loss can be weighted to obtain the target loss, and then the parameters of the encoder network and the decoder network can be adjusted by the second loss until the second training end condition is reached. In the above process, by introducing the divergence loss, the encoding features encoded by the encoder network can be made close to the first specified distribution, even if the encoding features encoded by the encoder network after training obey the first specified distribution.

[0190] See also Figure 8 , Figure 8 A fourth flow chart of the image generation method provided in the embodiment of the present application is given. Before step S120, the image generation method further includes steps S410-S450:

[0191] S410: Obtain a third sample object image and a third sample identity feature corresponding to the third sample object image, where the third sample identity feature is obtained by performing feature extraction on the third sample object image with reference to an object recognition model.

[0192] As above, the third sample object image may be the same as or different from the first sample object image and the second sample object image, which is not specifically limited here. The third sample object image may also be obtained from the data set mentioned above.

[0193] S420 , performing feature mapping on the identity features of the third sample by a distribution mapping network to obtain predicted distribution parameters.

[0194] Among them, the prediction distribution parameters refer to parameters that can characterize the distribution shape; taking the characteristic distribution as Gaussian distribution as an example, the corresponding prediction distribution parameters may include mean and variance.

[0195] In some implementations, the distribution mapping network may be a downsampling network, which performs distribution mapping according to the third sample identity feature and outputs predicted distribution parameters.

[0196] In some embodiments, step S420 may specifically include: mapping the third sample identity feature to the coding feature distribution by a distribution mapping network to obtain a predicted distribution parameter, where the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding.

[0197] S430 , the driving feature generation model performs feature restoration based on the predicted distribution parameters to obtain a sample driving feature that obeys the first distribution.

[0198] Among them, feature restoration refers to the process of restoring the mapped features to the third sample identity features according to the predicted distribution parameters; it can be understood that there is a difference between the restored third sample identity features and the input third sample identity features. In order to facilitate the distinction, the restored third sample identity features are described as sample driving features.

[0199] In some embodiments, the driving feature generation model may perform upsampling processing on the predicted distribution parameters to obtain sample driving features that obey the first distribution.

[0200] S440: Determine a third loss based on the sample driving feature and the third sample identity feature.

[0201] Among them, the sample driving feature and the third sample identity feature can be calculated by the second loss function to obtain the third loss. The second loss function can be a mean square error loss function, a cross entropy loss function, a cosine similarity loss function, etc.

[0202] S450: Adjust the parameters of the driving feature generation model and the distribution mapping network according to the third loss until the third training end condition is reached.

[0203] Specifically, according to the third loss, based on the gradient descent method (such as stochastic gradient descent, stochastic gradient descent with momentum term, etc.), the parameters of the driving feature generation model and the distribution mapping network are adjusted to minimize the third loss; the above steps S410-S450 are repeated using the adjusted driving feature generation model and distribution mapping network until the third training end condition is reached.

[0204] The third training end condition may be that the third loss is less than a third threshold, or that the number of training iterations of the driving feature generation model and the distribution mapping network reaches a set third number threshold.

[0205] In some embodiments, a divergence constraint between the output of the distribution mapping network and the target feature distribution (target feature distribution such as a Gaussian distribution) may also be added at the output position of the distribution mapping network to constrain training of the distribution mapping network so that the output of the distribution mapping network approaches the target feature distribution.

[0206] Through the method provided in the embodiment of the present application, the driving feature generation model can restore any feature that satisfies the predicted distribution parameters to obtain the sample driving feature, and by minimizing the third loss between the sample driving feature and the third sample identity feature, the obtained sample driving feature is made sufficiently close to the first distribution obeyed by the third sample identity feature, thereby ensuring that after training, the driving feature generation model can transform the input feature into a feature that obeys the first distribution.

[0207] It can be understood that since the sampling features are encoded by the encoder network, the sampling features obey the encoding feature distribution. On this basis, mapping the third sample identity features to the encoding feature distribution can maintain the consistency of the driving feature generation model during the training process and the use process, and improve the use effect of the driving feature generation model.

[0208] The method of the present application is described in detail below with reference to a specific embodiment.

[0209] like Fig. 9 As shown in Figure 1, the training process of the object recognition model is divided into the model training phase and the object image application phase. Fig. 9 The model training stage shown includes the following process of step 911 to step 913: step 910, training the encoder network and the decoder network. Step 920, training the diffusion network. Step 930, training the driving feature generation model.

[0210] In the object image application stage, the object image is first generated by driving the feature generation model and the image generation model, and then the generated object image is used to train the object recognition model.

[0211] Among them, training the encoder network and decoder network can be as follows Fig.10 As shown, the following steps 1010-1060 are included:

[0212] Step 1010: Acquire second training data; wherein the second training data includes a second sample image.

[0213] Step 1020: the encoder network performs feature encoding; that is, the encoder network encodes the second sample image to obtain a second encoded feature.

[0214] Step 1030: the decoder network performs image decoding; that is, the decoder network decodes the second encoded feature to obtain a decoded sample object image.

[0215] Step 1040, calculating a second loss; specifically, performing loss calculation according to the second sample object image and the decoded sample object image to obtain the second loss.

[0216] Step 1050, determining whether the second training end condition is met;

[0217] If satisfied, end the training of encoder network and decoder network;

[0218] If not satisfied, execute step 1060 to adjust the parameters of the encoder network and the decoder network, and then return to step 1010.

[0219] The process of training the diffusion network can be as follows: Fig.11 As shown, including steps 1110-1191:

[0220] Step 1110, obtaining first training data; wherein the first training data includes a first sample image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image.

[0221] Step 1120: the encoder network performs feature encoding. Specifically, the first sample object image is encoded using the encoder network to obtain a first encoded feature, wherein the encoder network may be trained based on the second sample image.

[0222] Step 1130, the diffusion network performs diffusion denoising; specifically, the diffusion network performs k rounds of diffusion denoising on the first coding feature to obtain a diffusion denoising result; the diffusion denoising result includes the target sample denoising feature obtained by the k-th round of diffusion denoising; then, using the first sample identity feature as a diffusion condition, the target sample denoising feature is subjected to k rounds of diffusion denoising to obtain a sample denoising feature.

[0223] Step 1140, calculating the diffusion loss; specifically, determining the diffusion loss according to the noise added in the process of performing k rounds of diffusion denoising on the first coding feature and the noise predicted in the process of performing k rounds of diffusion denoising on the target sample noisy feature.

[0224] Step 1150: the decoder network performs image decoding; specifically, the sample denoising features are decoded using the decoder network to obtain an object decoded image, wherein the decoder network may be trained based on the second sample image.

[0225] Step 1160 , performing identity recognition with reference to the object recognition model; specifically, inputting the object decoded image into the reference object recognition model, and extracting the identity features of the object decoded image with reference to the object recognition model to obtain reference identity features.

[0226] Step 1170, calculating the identity feature constraint loss; specifically, calculating the identity feature constraint loss based on the reference identity feature and the first sample identity feature provided in the first training data.

[0227] Step 1180, calculating the first loss; specifically, performing weighted summation on the diffusion loss and the identity feature constraint loss to obtain the first loss.

[0228] Step 1190, determine whether the first training end condition is met; if satisfied, end the training of the diffusion network; if not satisfied, execute step 1191, adjust the parameters of the diffusion network, and then return to step 1110.

[0229] The process of training the driving feature generation model can be as follows Fig.12 As shown, steps 1210-1270 are included:

[0230] Step 1210: Acquire a third sample image.

[0231] Step 1220, the object recognition model extracts identity features; specifically, the third sample image features are input into the object recognition model, and the reference object recognition model extracts features from the third sample image to obtain the third sample identity features.

[0232] Step 1230: the distribution mapping network performs distribution mapping; specifically, the third sample identity feature is distributed mapped by the distribution mapping network to obtain a predicted distribution parameter. The distribution mapping network may implement distribution mapping of the third sample identity feature by feature downsampling.

[0233] Step 1240: The driving feature generation model performs feature restoration; specifically, the driving feature generation model performs feature restoration according to the predicted distribution parameters to obtain the sample driving feature.

[0234] The driving feature generation model can restore the predicted distribution parameters to sample driving features through feature upsampling.

[0235] Step 1250, calculate the third loss; specifically, calculate the third loss based on the sample driving feature output by the feature upsampling network and the third sample identity feature output by the object recognition model.

[0236] Step 1260, determining whether the third training end condition is met;

[0237] If the third training end condition is met, determining to end the training of the distribution mapping network and the driving feature generation model;

[0238] If the third training end condition is not met, step 1270 is executed to adjust the parameters of the distribution mapping network and the driving feature generation model, and the above training steps are repeated until the third training end condition is met.

[0239] Finally, it should be noted that in the above embodiment, the first sample object image used for image generation model training, the second sample object image used for encoder network and decoder network training, and the third sample object image used for driving feature generation model training belong to the same sample training set, which can be, for example, the data set mentioned above.

[0240] It can be understood that since the image generation model, encoder network, decoder network and driving feature generation model are used in different stages of the same image generation process, training multiple models using images in the same sample training set can ensure the consistency of training of multiple models and improve the training effect of the model.

[0241] For easier understanding, please refer to Fig.13 , Fig.13 An application flow chart of the image generation method provided in the embodiment of the present application is provided, including steps 1310-1370:

[0242] Step 1310, obtaining sampling features, wherein the sampling features may refer to features obtained by random sampling from the image coding feature space. The feature distribution obeyed by the sampling features may be a Gaussian distribution.

[0243] Step 1320 , the driving feature generation model outputs the target identity feature; specifically, the sampling feature is input into the driving feature generation model, and the driving feature generation model performs feature mapping on the sampling feature to obtain a first identity feature that obeys a first distribution.

[0244] Step 1330, input a randomly initialized noise image.

[0245] Step 1340: the image generation model generates an object image. Specifically, the target identity feature output by the driving feature generation model and the noise image are input into the image generation model, and the image generation model performs diffusion denoising on multiple different noise images according to the first identity feature to obtain multiple different object images of the identity represented by the first identity feature.

[0246] Step 1350: Output the object image. The output object image is used as an input of the object recognition model and can be used for training the object recognition model.

[0247] Among them, in step 1340, the image generation model may include a diffusion network and a decoder network. The first identity feature and the randomly initialized noise image are first input into the diffusion network. The diffusion network uses the first identity feature as a diffusion condition to perform multiple rounds of diffusion denoising on the noise image. After the number of rounds of diffusion denoising reaches a preset number, the target denoising feature output by the last round of diffusion denoising is output. The decoder network decodes based on the output target denoising feature to obtain the object image.

[0248] Furthermore, the diffusion network may also include a diffusion denoising module, a decoding module, and an image adjustment module; wherein the diffusion denoising module is used to perform multiple rounds of diffusion denoising on the noisy image; the decoding module may be a decoder network, and the decoding module may obtain the noise image that needs to be removed in this round of diffusion denoising by decoding the output of the diffusion denoising module; the image adjustment module denoises the initialized noise image according to the decoded noise image to obtain the denoising features of the denoised image output by this round of diffusion denoising, and if the denoising rounds reach the preset rounds, the target denoising features of the output of this round of diffusion denoising are output, and the encoder network decodes based on the output target denoising features to obtain the object image; if the denoising rounds do not reach the preset rounds, the denoising features of the denoised image output by this round of diffusion denoising are used as the input of the next round of diffusion denoising module.

[0249] In some embodiments, the object recognition model is a face recognition model, and the object image is a face image; in other embodiments, the object recognition model is an iris recognition model, and the object image is an iris image.

[0250] Based on the image generation method provided in the above embodiment, the present application also provides a method for training an object recognition model. The method for training an object recognition model includes the following steps a and b:

[0251] Step a: obtaining training data, where the training data includes multiple different object images with the same identity identifier, and the multiple different object images with the same identity identifier are generated according to the image generation method provided in this application.

[0252] Step b: iteratively train the object recognition model based on multiple different object images with the same identity identification until the fourth training end condition is reached.

[0253] Through the training method of the object recognition model provided in the embodiment of the present application, the object recognition model can not only recognize object images corresponding to different identity identifiers, but also recognize different object images corresponding to the same identity identifier, thereby enhancing the recognition effect of the object recognition model.

[0254] In some embodiments, the trained object recognition model can be applied to payment scenarios, that is, the object recognition model is used to perform object recognition on the collected object images (such as face images, palm print images, fingerprint images, iris images, etc.) to obtain a recognition result of the object image. The recognition result indicates the identity of the object presented in the object image. Afterwards, payment processing is performed based on the payment account bound to the identity indicated by the recognition result.

[0255] In some embodiments, the trained object recognition model can be applied to access control scenarios. The object recognition model is used to perform object recognition on the collected object image to obtain a recognition result of the object image. The recognition result indicates the identity of the object presented in the object image. Then, based on the identity indicated by the recognition result, it is determined whether the access control permission is available. If it is determined that the identity indicated by the recognition result has the access control permission, the gate can be controlled to release the access control, or the smart door lock can be controlled to unlock the door.

[0256] Taking the object recognition model as a face recognition model as an example, the recognition effect of the face recognition model is shown in Table 1 below:

[0257] Table 1

[0258]

[0259] Among them, lfw, Cfpfp, Agedb, Cplfw and Calfw are commonly used data test sets in the field of face recognition. The data under each data test set represents the test accuracy under the data test set, and avg represents the average value. The specific explanations of each data test set are as follows:

[0260] LFW (Labeled Faces in the Wild), unrestricted labeled faces, is used to evaluate face recognition tasks in unconstrained environments.

[0261] CFP (Celebrities in Frontal Profile) is a specific face dataset that focuses on frontal photos and image photos of celebrities.

[0262] AgeDB (Age Database) is an age dataset containing facial images of people at different ages. It is usually used to evaluate the performance of face recognition algorithms under different age changes.

[0263] CPLFW (Cross-Pose LFW), unrestricted cross-pose labeled faces, CPLFW is a modified version of the LFW dataset, designed to evaluate the performance of cross-pose face verification in an unconstrained environment; this dataset further expands the intra-class variance by increasing pose differences, and deliberately selects negative pairs to avoid different genders.

[0264] CALFW (Cross-Age LFW), unrestricted cross-age labeled faces, CALFW is a modified version of the LFW dataset, which aims to evaluate the performance of cross-age face verification in an unconstrained environment; this dataset further expands the intra-class variance by increasing age differences, which helps to study the stability and accuracy of face recognition algorithms under different age changes.

[0265] Among them, the related technology adopts a random sampling method to generate an object image, that is, the image generation model is driven according to the randomly sampled features to generate the object image, and the generated object image is used to train the face recognition model; while the present application uses a driven feature generation model to perform feature mapping on the randomly sampled sampling features to obtain a first identity feature that obeys a first distribution, drives the image generation model according to the first identity feature to generate the object image, and then uses the generated object image to train the face recognition model, wherein the first distribution is the feature distribution obeyed by the identity feature extracted by the face recognition model.

[0266] It can be seen from the above table that compared with the face recognition model trained by object images generated by random sampling, hereinafter referred to as the randomly sampled face recognition model, the face recognition model trained by object images generated by the method provided in the present application has a better recognition accuracy in each data set than the randomly sampled face recognition model, that is, the face recognition model trained by the present application has a better recognition effect.

[0267] Based on the image generation method provided in the above embodiment, the present application also provides an image generation device, see Fig.14 , Fig.14 A schematic diagram of an image generating device provided in an embodiment of the present application is given, and the image generating device 500 includes:

[0268] The first acquisition module 510 is used to acquire sampling features.

[0269] The mapping module 520 is used to perform feature mapping on the sampled features by the driving feature generation model to obtain a first identity feature that obeys a first distribution. The first distribution is a feature distribution obeyed by the identity feature extracted by the object recognition model.

[0270] The image generation module 530 is used to drive the image generation model to perform diffusion denoising on multiple different noise images through the first identity feature, so as to obtain multiple different object images of the identity represented by the first identity feature. The multiple different object images of the identity represented by the first identity feature are used to train the object recognition model.

[0271] In some embodiments, the image generation model includes a diffusion network and a decoder network; the image generation module 530 includes a denoising unit and a decoding unit. For each noise image in a plurality of different noise images, the denoising unit is used to use the first identity feature as a diffusion condition by the diffusion network to perform T rounds of diffusion denoising on the noise image to obtain a target denoising feature output by the T round of diffusion denoising; wherein the denoising feature output by the i-th round of diffusion denoising is used as an input for the i+1-th round of diffusion denoising, i and T are positive integers, i<T; the decoding unit is used to decode the target denoising feature by the decoder network to obtain an object image represented by the first identity feature.

[0272] In some embodiments, the image generating device 500 further includes a first training preparation module, which is used to obtain a first sample object image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image, wherein the first sample identity feature is obtained by extracting features from the first sample object image using a reference object recognition model, and the reference object recognition model is an object recognition model or other models with the same recognition task as the object recognition model; k is a positive integer not exceeding T; a first encoding module, which is used to encode the first sample object image using an encoder network to obtain a first encoding feature; a first denoising module, which is used to perform k rounds of diffusion denoising on the first encoding feature using a diffusion network to obtain a diffusion denoising result; and the diffusion denoising result includes: The invention comprises a target sample denoising feature obtained by the k-th round of diffusion denoising; a first denoising module, which is used to perform k rounds of diffusion denoising on the target sample denoising feature with the first sample identity feature as the diffusion condition by the diffusion network to obtain the sample denoising feature; a first decoding module, which is used to decode the sample denoising feature by the decoder network to obtain the object decoded image; a first extraction module, which is used to extract the identity feature of the object decoded image by the reference object recognition model to obtain the reference identity feature corresponding to the sample denoising feature; a first loss calculation module, which is used to determine the identity feature constraint loss according to the first sample identity feature and the reference identity feature; a first parameter adjustment module, which adjusts at least the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

[0273] In some embodiments, the first loss calculation module includes a compensation coefficient determination unit, which is used to determine the compensation coefficient with reference to round k, and the compensation coefficient is negatively correlated with the reference round k; a first calculation unit, which is used to calculate the feature loss between the first sample identity feature and the reference identity feature; and a compensation unit, which is used to compensate for the feature loss through the compensation coefficient to obtain the identity feature constraint loss.

[0274] In some embodiments, the diffusion denoising result also includes the noise predicted in the k-round diffusion denoising process, and the image generating device 500 also includes a diffusion loss calculation module, which is used to determine the diffusion loss based on the noise added by the k-round diffusion denoising of the first coding feature and the noise predicted in the process of performing k-round diffusion denoising on the target sample noisy feature; the first parameter adjustment module also includes a weighted calculation unit, which is used to weight the identity feature constraint loss and the diffusion loss to obtain a first loss; and an adjustment unit, which is used to adjust at least the parameters of the diffusion network according to the first loss until the first training end condition is reached.

[0275] In some embodiments, the first parameter adjustment module is further used to fix parameters of the encoder network and the decoder network, and adjust parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

[0276] In some embodiments, the image generating device 500 further includes a second training preparation module for acquiring a second sample object image; a second encoding module for performing feature extraction on the second sample object image by an encoder network to obtain a second encoded feature; a second decoding module for decoding the second encoded feature by a decoder network to obtain a decoded sample object image; a second loss calculation module for determining a second loss based on the second sample object image and the decoded sample object image; and a second parameter adjustment module for adjusting the parameters of the encoder network and the decoder network based on the second loss until a second training end condition is reached.

[0277] In some implementations, the first acquisition module 510 is used to perform sampling in the coding feature distribution to obtain the sampled features; the coding feature distribution refers to the distribution obeyed by the coding features obtained by the encoder network encoding.

[0278] In some embodiments, the image generating device 500 further includes a third training preparation module, which is used to obtain a third sample object image and a third sample identity feature corresponding to the third sample object image, where the third sample identity feature is obtained by extracting features from the third sample object image with reference to the object recognition model; a second mapping module, which is used to map the third sample identity feature to the coding feature distribution by a distribution mapping network to obtain predicted distribution parameters; the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding; a feature restoration module, which is used to perform feature restoration based on the predicted distribution parameters by the driving feature generation model to obtain sample driving features that obey the first distribution; a third loss calculation module, which is used to determine a third loss based on the sample driving feature and the third sample identity feature; and a third parameter adjustment module, which is used to adjust the parameters of the driving feature generation model and the distribution mapping network according to the third loss until the third training end condition is reached.

[0279] In some implementations, the object recognition model is a face recognition model, and the object image is a face image.

[0280] In some embodiments, based on the object recognition model training method provided in the above embodiments, the present application also provides an object recognition model training device, see Fig.15 , Fig.15 A schematic diagram of a training device for an object recognition model provided in an embodiment of the present application is provided. The training device 600 for an object recognition model includes:

[0281] A second acquisition module 610 is used to acquire training data, where the training data includes a plurality of different object images with the same identity, where the plurality of different object images with the same identity are generated by the image generation method provided by the above embodiment;

[0282] The object recognition model training module 620 is used to iteratively train the object recognition model according to a plurality of different object images with the same identity, until a fourth training end condition is reached.

[0283] Fig.16 The schematic diagram of the structure of a computer system of an electronic device suitable for implementing the embodiment of the present application is shown. The electronic device may be a terminal as described above, which is used to implement the image generation method or the object recognition model training method provided by the present application. It should be noted that: Fig.16 The computer system 1300 of the electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0284] like Fig.16 As shown, the computer system 1600 includes a central processing unit (CPU) 1601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 1602 or the program loaded from the storage part 1608 to the random access memory (RAM) 1603, such as executing the method in the above embodiment. Various programs and data required for system operation are also stored in the RAM 1603. The CPU 1601, ROM 1602 and RAM 1603 are connected to each other through a bus 1604. The input / output (I / O) interface 1605 is also connected to the bus 1604.

[0285] The following components are connected to the I / O interface 1605: an input section 1606 including a keyboard, a mouse, a microphone, etc.; an output section 1607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1608 including a hard disk, etc.; and a communication section 1609 including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section 1609 performs communication processing via a network such as the Internet. A drive 1610 is also connected to the I / O interface 1605 as needed. A removable medium 1611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 1610 as needed so that the computer instructions read therefrom are installed into the storage section 1608 as needed.

[0286] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes computer instructions. When the computer instructions are executed by the central processing unit (CPU) 1601, various functions defined in the system of the present application are executed.

[0287] The present application also provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed by a processor, the method in any of the above method embodiments is implemented.

[0288] It should be noted that the computer-readable storage medium shown in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, device or device. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0289] In the embodiments of the present application, the term "module" or "unit" refers to a computer instruction or a part of a computer instruction with a predetermined function, and works together with other related parts to achieve a predetermined goal, and can be implemented in whole or in part by using software, hardware (such as processing circuits or memories) or a combination thereof. Similarly, a processor (or multiple processors or memories) can be used to implement one or more modules or units. In addition, each module or unit can be a part of the overall module or unit of the module or unit function.

[0290] The above are only preferred embodiments of the present application, and are not intended to limit the present application in any form. Although the present application has been disclosed as above with preferred embodiments, it is not intended to limit the present application. Any technical personnel in the field can make some changes or modify the technical contents disclosed above into equivalent embodiments without departing from the scope of the technical solution of the present application. However, any brief modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present application without departing from the content of the technical solution of the present application are still within the scope of the technical solution of the present application.

Claims

1. An image generation method, characterized in that: include: Acquire sampling features; wherein the sampling features are obtained by sampling from a coding feature distribution, or the sampling features are image features extracted from a reference object image by a trained encoder network, and the coding feature distribution refers to the distribution obeyed by the coding features obtained by encoding the encoder network; The driving feature generation model performs feature mapping on the sampled features to obtain a first identity feature that obeys a first distribution, where the first distribution is a feature distribution obeyed by the identity feature extracted by the object recognition model; Through the first identity feature, the image generation model is driven to perform diffusion denoising on multiple different noise images respectively, so as to obtain multiple different object images of the identity represented by the first identity feature. The multiple different object images of the identity represented by the first identity feature are used to train the object recognition model.

2. The method according to claim 1, characterized in that The image generation model includes a diffusion network and a decoder network; The method of driving the image generation model to perform diffusion denoising on a plurality of different noise images respectively through the first identity feature to obtain a plurality of different object images represented by the identity feature, includes: The following processing is performed on each of the multiple different noise images: The diffusion network uses the first identity feature as a diffusion condition to perform T rounds of diffusion denoising on the noisy image to obtain a target denoising feature output by the T-th round of diffusion denoising; wherein the denoising feature output by the i-th round of diffusion denoising is used as an input for the i+1-th round of diffusion denoising, i and T are positive integers, and i<T; The target denoising feature is decoded by the decoder network to obtain an object image of the identity represented by the first identity feature.

3. The method according to claim 2, characterized in that Before the method drives the image generation model to perform diffusion denoising on a plurality of different noise images respectively by using the first identity feature to obtain a plurality of different object images of the identity represented by the first identity feature, the method further includes: Obtaining a first sample object image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image, wherein the first sample identity feature is obtained by extracting features from the first sample object image using a reference object recognition model, wherein the reference object recognition model is the object recognition model or another model having the same recognition task as the object recognition model; k is a positive integer not exceeding T; The encoder network performs encoding processing on the first sample object image to obtain a first encoding feature; The diffusion network performs k rounds of diffusion noise addition on the first coding feature to obtain a diffusion noise addition result; the diffusion noise addition result includes a target sample noise addition feature obtained by the k-th round of diffusion noise addition; The diffusion network uses the first sample identity feature as a diffusion condition to perform k rounds of diffusion denoising on the target sample noise feature to obtain a sample denoising feature; The decoder network decodes the sample denoising features to obtain an object decoded image; The reference object recognition model extracts identity features from the decoded object image to obtain reference identity features corresponding to the sample denoising features; Determining an identity feature constraint loss according to the first sample identity feature and the reference identity feature; According to the identity feature constraint loss, at least the parameters of the diffusion network are adjusted until a first training end condition is reached.

4. The method according to claim 3, characterized in that The determining the identity feature constraint loss according to the first sample identity feature and the reference identity feature includes: Determine a compensation coefficient according to the reference round k, wherein the compensation coefficient is negatively correlated with the reference round k; Calculating a feature loss between the first sample identity feature and the reference identity feature; The feature loss is compensated by the compensation coefficient to obtain the identity feature constraint loss.

5. The method according to claim 3, characterized in that: The diffusion denoising result also includes the noise predicted in the k-round diffusion denoising process; The method further comprises: Determine diffusion loss based on the noise added during k rounds of diffusion denoising on the first coding feature and the noise predicted during k rounds of diffusion denoising on the target sample noisy feature; The step of at least adjusting the parameters of the diffusion network according to the identity feature constraint loss until a first training end condition is reached includes: Performing weighted processing on the identity feature constraint loss and the diffusion loss to obtain a first loss; According to the first loss, at least a parameter of the diffusion network is adjusted until a first training end condition is reached.

6. The method according to claim 3, characterized in that The step of at least adjusting the parameters of the diffusion network according to the identity feature constraint loss until a first training end condition is reached includes: The parameters of the encoder network and the decoder network are fixed, and the parameters of the diffusion network are adjusted according to the identity feature constraint loss until a first training end condition is reached.

7. The method according to claim 3, characterized in that Before the encoder network encodes the first sample object image to obtain the first encoding feature, the method includes: acquiring a second sample object image; Extracting features from the second sample object image by the encoder network to obtain second encoding features; Decoding the second encoded feature by the decoder network to obtain a decoded sample object image; determining a second loss based on the second sample object image and the decoded sample object image; Parameters of the encoder network and the decoder network are adjusted according to the second loss until a second training end condition is reached.

8. The method according to any one of claims 3 to 7, characterized in that Before the driving feature generation model performs feature mapping on the sampling feature to obtain a first identity feature that obeys a first distribution, the method further includes: Acquire a third sample object image and a third sample identity feature corresponding to the third sample object image, wherein the third sample identity feature is obtained by extracting features from the third sample object image by the reference object recognition model; The third sample identity feature is mapped to the coding feature distribution by a distribution mapping network to obtain a predicted distribution parameter; the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding; The driving feature generation model performs feature restoration based on the predicted distribution parameters to obtain a sample driving feature that obeys the first distribution; determining a third loss based on the sample driving feature and the third sample identity feature; According to the third loss, the parameters of the driving feature generation model and the distribution mapping network are adjusted until a third training end condition is reached.

9. The method according to claim 1, characterized in that: The object recognition model is a face recognition model, and the object image is a face image.

10. A method for training an object recognition model, characterized in that: include: Acquire training data, wherein the training data includes a plurality of different object images with the same identity, wherein the plurality of different object images with the same identity are generated according to the method as described in any one of claims 1 to 9; The object recognition model is iteratively trained according to a plurality of different object images with the same identity, until a fourth training end condition is reached.

11. An image generating device, characterized in that: include: A first acquisition module is used to acquire sampling features; wherein the sampling features are obtained by sampling from a coding feature distribution, or the sampling features are image features extracted from a reference object image by a trained encoder network, and the coding feature distribution refers to the distribution obeyed by the coding features obtained by encoding the encoder network; A mapping module, configured to perform feature mapping on the sampled features by a driving feature generation model to obtain a first identity feature that obeys a first distribution, wherein the first distribution is a feature distribution obeyed by the identity feature extracted by the object recognition model; An image generation module is used to drive an image generation model to perform diffusion denoising on multiple different noise images respectively through the first identity feature to obtain multiple different object images represented by the identity feature, and the multiple different object images represented by the first identity feature are used to train the object recognition model.

12. The device according to claim 11, characterized in that The image generation model includes a diffusion network and a decoder network; the image generation module includes a denoising unit and a decoding unit, for each of the multiple different noise images: A denoising unit, configured to use the diffusion network to perform T rounds of diffusion denoising on the noisy image using the first identity feature as a diffusion condition, and obtain a target denoising feature output by the T-th round of diffusion denoising; wherein the denoising feature output by the i-th round of diffusion denoising is used as an input for the i+1-th round of diffusion denoising, i and T are positive integers, and i<T; A decoding unit is used to decode the target denoising feature by the decoder network to obtain an object image represented by the identity feature.

13. The device according to claim 12, characterized in that The image generating device further comprises: a first training preparation module, configured to obtain a first sample object image, a first sample identity feature corresponding to the first sample object image, and a reference round k determined for the first sample object image, wherein the first sample identity feature is obtained by extracting features from the first sample object image using a reference object recognition model, wherein the reference object recognition model is the object recognition model or another model having the same recognition task as the object recognition model; k is a positive integer not exceeding T; A first encoding module, configured to perform encoding processing on the first sample object image by an encoder network to obtain a first encoding feature; A first noising module, configured to perform k rounds of diffusion noising on the first coding feature by the diffusion network to obtain a diffusion noising result; the diffusion noising result includes a target sample noising feature obtained by the k-th round of diffusion noising; A first denoising module, configured to use the diffusion network to perform k rounds of diffusion denoising on the target sample noise feature using the first sample identity feature as a diffusion condition to obtain a sample denoising feature; A first decoding module, configured to decode the sample denoising feature by the decoder network to obtain an object decoded image; A first extraction module, configured to extract identity features from the object decoded image using the reference object recognition model to obtain reference identity features corresponding to the sample denoising features; A first loss calculation module, configured to determine an identity feature constraint loss according to the first sample identity feature and the reference identity feature; The first parameter adjustment module adjusts at least the parameters of the diffusion network according to the identity feature constraint loss until a first training end condition is reached.

14. The device according to claim 13, characterized in that The first loss calculation module includes: a compensation coefficient determination unit, used for determining a compensation coefficient for the reference round k, wherein the compensation coefficient is negatively correlated with the reference round k; A first calculation unit, configured to calculate a feature loss between the first sample identity feature and the reference identity feature; The compensation unit is used to compensate the feature loss by using the compensation coefficient to obtain the identity feature constraint loss.

15. The device according to claim 13, characterized in that The diffusion denoising result also includes the noise predicted in the k-round diffusion denoising process, and the image generating device also includes: A diffusion loss calculation module, used to determine the diffusion loss according to the noise added in the process of performing k rounds of diffusion denoising on the first coding feature and the noise predicted in the process of performing k rounds of diffusion denoising on the target sample noisy feature; The first parameter adjustment module includes: A weighted calculation unit, used for performing weighted processing on the identity feature constraint loss and the diffusion loss to obtain a first loss; An adjustment unit is used to adjust at least a parameter of the diffusion network according to the first loss until a first training end condition is reached.

16. The device according to claim 13, characterized in that The first parameter adjustment module is also used to fix the parameters of the encoder network and the decoder network, and adjust the parameters of the diffusion network according to the identity feature constraint loss until the first training end condition is reached.

17. The device according to claim 13, characterized in that The image generating device further comprises: A second training preparation module, used for acquiring a second sample object image; A second encoding module, configured to extract features from the second sample object image by the encoder network to obtain second encoding features; A second decoding module, configured to decode the second encoded feature by the decoder network to obtain a decoded sample object image; A second loss calculation module, configured to determine a second loss according to the second sample object image and the decoded sample object image; A second parameter adjustment module is used to adjust the parameters of the encoder network and the decoder network according to the second loss until a second training end condition is reached.

18. The device according to any one of claims 13 to 17, characterized in that The image generating device further comprises: A third training preparation module, used for acquiring a third sample object image and a third sample identity feature corresponding to the third sample object image, wherein the third sample identity feature is obtained by extracting features from the third sample object image by the reference object recognition model; A second mapping module is used to map the third sample identity feature to a coding feature distribution by a distribution mapping network to obtain a predicted distribution parameter; the coding feature distribution refers to the distribution obeyed by the coding feature obtained by the encoder network encoding; A feature restoration module, configured to perform feature restoration based on the predicted distribution parameters by the driving feature generation model to obtain a sample driving feature that obeys the first distribution; A third loss calculation module, configured to determine a third loss based on the sample driving feature and the third sample identity feature; The third parameter adjustment module is used to adjust the parameters of the driving feature generation model and the distribution mapping network according to the third loss until a third training end condition is reached.

19. The device according to claim 11, characterized in that The object recognition model is a face recognition model, and the object image is a face image.

20. A training device for an object recognition model, characterized in that: include: A second acquisition module, configured to acquire training data, wherein the training data includes a plurality of different object images with the same identity, wherein the plurality of different object images with the same identity are generated according to the method of any one of claims 1 to 9; The object recognition model training module is used to iteratively train the object recognition model according to multiple different object images with the same identity identification until a fourth training end condition is reached.

21. An electronic device, characterized in that: include: processor; A memory, wherein computer instructions are stored in the memory, and when the computer instructions are executed by the processor, the method according to any one of claims 1 to 9 or 10 is implemented.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 9 or 10 is implemented.

23. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by a processor, the method described in any one of claims 1-9 or 10 is implemented.

Images