Image generation method and device, storage medium and electronic equipment

Abstract

The invention discloses an image generation method and device, a storage medium and electronic equipment. The method comprises the steps that the identity feature of the ith object in K objects is obtained, the ith identity feature is obtained, the ith identity feature is used for representing identity information of the ith object, and K is a positive integer larger than or equal to 2; m initial features of the ith object are obtained, each initial feature in the m initial features is used for representing one kind of object information of the ith object, and the object information comprises identity information and attribute information; obtaining m similarity thresholds of the ith object; and based on the ith identity feature and the m similarity thresholds, performing multi-round iterative adjustment on the m initial features to obtain m optimized features. According to the invention, the technical problem that the sample image for generating the training image recognition model is relatively single in the prior art is solved.

Description

Technical Field

[0001] This application relates to the field of image processing, and more specifically, to an image generation method and apparatus, a storage medium, and an electronic device. Background Technology

[0002] In the training process of image recognition algorithms (e.g., face recognition algorithms), a large number of real face images are usually required as training samples. However, considering privacy protection issues, some face datasets in existing face databases are unavailable, which brings great difficulties to the research of face recognition algorithms.

[0003] To address this issue, using synthetic face datasets has become an alternative solution for protecting user privacy during the development of face recognition algorithm models. Specifically, this involves obtaining facial features from a high-dimensional space through random sampling, and then using these randomly sampled facial features to drive the generation of face images.

[0004] However, since randomly sampled facial features are not screened or labeled, there may be repeated sampling of facial features from different identities, or mixing facial features from other identities into facial features from the same identity. This results in a lack of diversity between facial images generated using the sampled facial features. In other words, related technologies suffer from a problem of generating relatively homogeneous sample images for training facial recognition models.

[0005] There is currently no effective solution to the above problems. Summary of the Invention

[0006] This application provides an image generation method and apparatus, storage medium and electronic device to at least solve the technical problem of relatively simple sample images for generating training image recognition models.

[0007] According to one aspect of the embodiments of this application, an image generation method is provided, comprising: obtaining an identity feature of the i-th object from K objects to obtain an i-th identity feature, wherein the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K; obtaining m initial features of the i-th object, wherein each of the m initial features is used to characterize a kind of object information of the i-th object, the object information including identity information and attribute information, and m is a positive integer greater than or equal to 2; obtaining m similarity thresholds of the i-th object; and performing multiple rounds of iterative adjustment on the m initial features based on the i-th identity feature and the m similarity thresholds to obtain m optimized features, wherein, in the multiple rounds of iterative adjustment, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized, different optimized features in the m optimized features characterize different attribute information, and different optimized features in the m optimized features are used to generate different images of the i-th object.

[0008] Optionally, obtaining the m similarity thresholds of the i-th object includes: sampling m similarity thresholds within a preset range of similarity threshold values.

[0009] Optionally, obtaining the m similarity thresholds for the i-th object includes selecting m similarity thresholds that conform to a Gaussian distribution.

[0010] Optionally, minimizing the differences between the m similarities and m similarity thresholds between the aforementioned m optimized features and the i-th identity feature includes: minimizing the average value of the m differences when the m similarities include the similarity between each of the m optimized features and the i-th identity feature, wherein the m differences include the differences between the similarity between the m similarities corresponding to the j-th optimized feature and the similarity thresholds between the m similarities and the j-th optimized feature, where j is a positive integer less than or equal to m, and the m similarities and m optimized features have a one-to-one correspondence, and the m similarity thresholds and m optimized features have a one-to-one correspondence; or minimizing the mean square error of the m differences when the m similarities include the similarity between each of the m optimized features and the i-th identity feature.

[0011] Optionally, the above-mentioned multi-round iterative adjustment of m initial features based on the i-th identity feature and m similarity thresholds to obtain m optimized features includes: performing p-th round iterative adjustment on the m initial features through the following steps, where p is a positive integer: obtaining the similarity between each feature in the m features after the (p-1)-th round of iterative adjustment and the i-th identity feature, obtaining the m similarities in the p-th round, where when p equals 1, the m features after the (p-1)-th round of iterative adjustment are the m initial features; determining the m differences in the p-th round based on the m similarities and m similarity thresholds in the p-th round, where the m differences in the p-th round include the j-th similarity and the m similarity thresholds in the p-th round. The difference between the j-th similarity thresholds in the similarity thresholds, where j is a positive integer less than or equal to m, is used. There is a one-to-one correspondence between the m similarities in the p-th round and the m features adjusted in the (p-1)-th round. Based on the m differences in the p-th round, the similarity loss value for the p-th round is determined. If the similarity loss value does not meet the preset first end-of-adjustment condition, the m features adjusted in the (p-1)-th round are adjusted to obtain the m features adjusted in the p-th round. If the similarity loss value meets the first end-of-adjustment condition, the m features adjusted in the (p-1)-th round are determined as the m optimized features.

[0012] Optionally, determining the similarity loss value for round p based on the m differences in round p includes: determining the similarity loss value for round p as equal to the average of the m differences in round p; or determining the similarity loss value for round p as equal to the mean squared error of the m differences in round p.

[0013] Optionally, adjusting the m features after the (p-1)th iteration to obtain the m features after the p-th iteration includes: reducing at least some features of the m features after the (p-1)th iteration to obtain the m features after the p-th iteration; or reducing a first part of the features of the m features after the (p-1)th iteration and adding a second part of the features of the m features after the (p-1)th iteration to obtain the m features after the p-th iteration, wherein the first part of the features is different from the second part of the features.

[0014] Optionally, obtaining the identity feature of the i-th object among the K objects to obtain the i-th identity feature includes: selecting the identity feature of the i-th object from the K identity features to obtain the i-th identity feature, wherein the K identity features include the identity feature of each object among the K objects, and each identity feature in the K identity features is used to represent the identity information of the corresponding object among the K objects; or selecting the identity feature of the i-th object from the K identity features and normalizing the identity feature of the i-th object to obtain the i-th identity feature.

[0015] Optionally, the above method further includes: obtaining the initial identity features of each of the K objects to obtain K initial identity features; performing multiple rounds of iterative adjustments on the K initial identity features to obtain K identity features.

[0016] Optionally, obtaining the initial identity features of each of the K objects to obtain K initial identity features includes: sampling the initial identity features of the K objects from the preset M initial identity features to obtain K initial identity features, wherein the M initial identity features include the initial identity features of each of the M objects, and M is a positive integer greater than K.

[0017] Optionally, the above-mentioned process of performing multiple rounds of iterative adjustment on the K initial identity features to obtain K identity features includes: performing multiple rounds of iterative adjustment on the K initial identity features to obtain K identity features, wherein the loss value determined based on the K identity features is the minimum during the multiple rounds of iterative adjustment.

[0018] Optionally, the above-mentioned process of performing multiple rounds of iterative adjustment on the K initial identity features to obtain K identity features includes: performing the q-th round of iterative adjustment on the K initial identity features through the following steps, where q is a positive integer: determining the loss value of the q-th round based on the K features after the (q-1)-th round of iterative adjustment, wherein when q equals 1, the K features after the (q-1)-th round of iterative adjustment include the K initial identity features; if the loss value of the q-th round does not meet the preset second end adjustment condition, adjusting the K features after the (q-1)-th round of iterative adjustment to obtain the K features after the q-th round of iterative adjustment; if the loss value of the q-th round meets the second end adjustment condition, determining the K features after the (q-1)-th round of iterative adjustment as the K identity features.

[0019] Optionally, determining the loss value for round q based on the K features adjusted in round q-1 includes: multiplying the (q-1)th matrix with its transpose to obtain a first matrix, wherein the (q-1)th matrix comprises a matrix composed of the K features adjusted in round q-1; multiplying the identity matrix with preset coefficients to obtain a second matrix; subtracting the second matrix from the first matrix to obtain the loss value matrix for round q; and determining the loss value for round q based on the loss value matrix for round q.

[0020] Optionally, determining the loss value of round q based on the loss value matrix of round q includes: obtaining the largest eigenvalue in each row of the loss value matrix of round q to obtain K eigenvalues; and determining the loss value of round q as the average or mean squared error of the K eigenvalues.

[0021] Optionally, adjusting the K features after the (q-1)th iteration to obtain the K features after the qth iteration includes: reducing at least some features in the K features after the (q-1)th iteration to obtain the K features after the qth iteration; or reducing the third part of the K features after the (q-1)th iteration and adding the fourth part of the K features after the (q-1)th iteration to obtain the K features after the qth iteration, wherein the third part of the features is different from the fourth part of the features.

[0022] Optionally, after performing multiple rounds of iterative adjustments on the m initial features based on the i-th identity feature and m similarity thresholds to obtain m optimized features, the above method further includes: generating m candidate images of the i-th object based on each of the m optimized features and the initial noisy image.

[0023] Optionally, the above-mentioned generation of m candidate images of the i-th object based on each of the m optimized features and the initial noise image includes: generating one candidate image from the m candidate images by performing multiple rounds of iterative processing on one of the m optimized features and the initial noise image to obtain a candidate image.

[0024] Optionally, the above-mentioned process of performing multiple rounds of iterative processing on one of the m optimized features and the initial noise image to obtain a candidate image includes: performing the t-th round of iterative processing on one of the m optimized features and the initial noise image through the following steps, where t is a positive integer: denoising the noise image after the (t-1)-th round of iteration based on an optimized feature to obtain the first estimated noise image of the t-th round, where the noise image after the (t-1)-th round of iteration is the initial noise image when t=1; constraining the noise image after the (t-1)-th round of iteration based on an optimized feature to obtain the second estimated noise image of the t-th round; subtracting the first estimated noise image and the second estimated noise image of the t-th round from the noise image after the (t-1)-th round of iteration to obtain the noise image after the t-th round of iteration; and determining the noise image after the t-th round of iteration as a candidate image when t reaches a preset number of iterations.

[0025] According to another aspect of the embodiments of this application, an image generation method is also provided, comprising: obtaining m optimized features of an i-th object and an initial noise image, wherein the m optimized features are features obtained by iteratively adjusting the m initial features based on an i-th identity feature and m similarity thresholds, the i-th identity feature is used to characterize the identity information of the i-th object, i is a positive integer, each of the m initial features is used to characterize one type of object information of the i-th object, the object information includes identity information and attribute information, m is a positive integer greater than or equal to 2, in the multi-round iterative adjustment, the difference between the m similarities and the m similarity thresholds between the m optimized features and the i-th identity feature is minimized, different optimized features in the m optimized features characterize different attribute information, and different optimized features in the m optimized features are used to generate different images of the i-th object; and generating m candidate images of the i-th object based on each optimized feature in the m optimized features and the initial noise image.

[0026] Optionally, the above-mentioned generation of m candidate images of the i-th object based on each of the m optimized features and the initial noise image includes: generating one candidate image from the m candidate images by performing multiple rounds of iterative processing on one of the m optimized features and the initial noise image to obtain a candidate image.

[0027] Optionally, the above-mentioned process of performing multiple rounds of iterative processing on one of the m optimized features and the initial noise image to obtain a candidate image includes: performing the t-th round of iterative processing on one of the m optimized features and the initial noise image through the following steps, where t is a positive integer: denoising the noise image after the (t-1)-th round of iteration based on an optimized feature to obtain the first estimated noise image of the t-th round, where the noise image after the (t-1)-th round of iteration is the initial noise image when t=1; constraining the noise image after the (t-1)-th round of iteration based on an optimized feature to obtain the second estimated noise image of the t-th round; subtracting the first estimated noise image and the second estimated noise image of the t-th round from the noise image after the (t-1)-th round of iteration to obtain the noise image after the t-th round of iteration; and determining the noise image after the t-th round of iteration as a candidate image when t reaches a preset number of iterations.

[0028] According to another aspect of the embodiments of this application, an image generation apparatus is also provided, comprising: a first acquisition unit, configured to acquire the identity features of the i-th object among K objects to obtain the i-th identity feature, wherein the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K; a second acquisition unit, configured to acquire m initial features of the i-th object, wherein each of the m initial features is used to characterize a kind of object information of the i-th object, the object information including identity information and attribute information, and m is a positive integer greater than or equal to 2; a third acquisition unit, configured to acquire m similarity thresholds of the i-th object; and a first processing unit, configured to perform multiple rounds of iterative adjustment on the m initial features based on the i-th identity feature and the m similarity thresholds to obtain m optimized features, wherein in the multiple rounds of iterative adjustment, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized, different optimized features among the m optimized features characterize different attribute information, and different optimized features among the m optimized features are used to generate different images of the i-th object.

[0029] According to another aspect of the embodiments of this application, an image generation apparatus is also provided, comprising: a fourth acquisition unit, configured to acquire m optimized features of an i-th object and an initial noise image, wherein the m optimized features are features obtained by iteratively adjusting the m initial features based on an i-th identity feature and m similarity thresholds, the i-th identity feature is used to characterize the identity information of the i-th object, i is a positive integer, each of the m initial features is used to characterize one type of object information of the i-th object, the object information includes identity information and attribute information, m is a positive integer greater than or equal to 2, in the multi-round iterative adjustment, the difference between the m similarities and the m similarity thresholds between the m optimized features and the i-th identity feature is minimized, different optimized features in the m optimized features characterize different attribute information, and different optimized features in the m optimized features are used to generate different images of the i-th object; and a second processing unit, configured to generate m candidate images of the i-th object based on each optimized feature in the m optimized features and the initial noise image.

[0030] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored in the computer program for executing the above-described image generation method when the electronic device is run.

[0031] According to another aspect of the embodiments of this application, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0032] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program and the processor is configured to execute the image generation method described above through the computer program.

[0033] By iteratively adjusting the m initial features of the i-th object based on its identity features and m similarity thresholds, m optimized features are obtained. Specifically, the similarity between the m initial features and the i-th identity features is constrained to minimize the difference between the similarity to the m similarity thresholds, thus ensuring that the similarity of the adjusted m optimized features is constrained to around the target similarity. This allows multiple images of the i-th object to be generated using the m optimized features, increasing the diversity of sample images for the same object and improving the quality of sample images used to train the image recognition model. Attached Figure Description

[0034] The accompanying drawings, which are provided to further illustrate this application and form part of this application, illustrate exemplary embodiments of this application and are used to explain this application, but do not constitute an undue limitation of this application.

[0035] Figure 1 This is a schematic diagram illustrating an application scenario of an optional image generation method according to an embodiment of this application;

[0036] Figure 2 This is a flowchart of an optional image generation method according to an embodiment of this application;

[0037] Figure 3 This is an overall flowchart of an optional image generation method according to an embodiment of this application;

[0038] Figure 4 This is an overall flowchart of another optional image generation method according to an embodiment of this application;

[0039] Figure 5 This is a flowchart of an optional method for determining an inter-class identity feature matrix according to an embodiment of this application;

[0040] Figure 6 This is a flowchart of an optional method for determining the intra-class feature matrix according to an embodiment of this application;

[0041] Figure 7 This is a flowchart of an optional training process for a diffusion network according to an embodiment of this application;

[0042] Figure 8 This is a specific embodiment of an optional image generation method according to the embodiments of this application;

[0043] Figure 9 This is a schematic diagram of an optional first matrix according to an embodiment of this application;

[0044] Figure 10 This is a flowchart of another optional image generation method according to an embodiment of this application;

[0045] Figure 11 This is a schematic diagram of an optional image generation apparatus according to an embodiment of this application;

[0046] Figure 12 This is a schematic diagram of another optional image generation apparatus according to an embodiment of this application;

[0047] Figure 13 This is a schematic diagram of the structure of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0048] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0049] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0050] The technical solutions in this application will comply with legal regulations during implementation. When performing operations according to the technical solutions in the embodiments, the data used will not involve user privacy, ensuring that the operation process is compliant and legal while guaranteeing data security.

[0051] Definitions:

[0052] Intra-class data: Data of the same identity is classified as one class, and multiple images of the same identity are contained within the same identity;

[0053] Inter-class data: Data with different identities, that is, data between different IDs corresponding to different objects, is inter-class data;

[0054] Hypersphere: During the training of a face recognition model, it is assumed that the face features are distributed on a hypersphere space, which is a high-order manifold.

[0055] In addition, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant regulations and standards of the relevant countries or regions.

[0056] According to one aspect of the embodiments of this application, an image generation method is provided. As an optional implementation, the above-described image generation method can be applied, but is not limited to, to applications such as... Figure 1 The application scenarios shown are as follows. In, for example... Figure 1In the application scenario shown, the target terminal 102 can communicate with the server 106 via network 104, but is not limited to this. The server 106 can perform operations on the database 108, such as writing or reading data. The target terminal 102 may include, but is not limited to, a human-computer interaction screen, a processor, and a memory. The human-computer interaction screen may be used to display an initial noisy image, different images of the i-th object, etc., on the target terminal 102. The processor may be used to respond to the human-computer interaction operation, execute corresponding operations, or generate corresponding instructions and send the generated instructions to the server 106. The memory is used to store relevant processing data, such as the i-th identity feature, m initial features, and different images of the i-th object.

[0057] Optionally, in this embodiment, the target terminal can be a terminal configured with a target client, which may include, but is not limited to, at least one of the following: mobile phone (such as Android phone, iOS phone, etc.), laptop computer, tablet computer, PDA, MID (Mobile Internet Devices), PAD, desktop computer, smart TV, etc. The target client may be a video client, instant messaging client, browser client, educational client, etc. The network may include, but is not limited to, wired network and wireless network, wherein the wired network includes: local area network, metropolitan area network and wide area network, and the wireless network includes: Bluetooth, WIFI and other networks that enable wireless communication. The server may be a single server, a server cluster composed of multiple servers, or a cloud server.

[0058] The technical solutions in this application embodiment can be applied, but are not limited to, scenarios such as obtaining training data for face recognition models. They are mainly based on the identity features of the i-th object and m similarity thresholds, to optimize the m initial features of the i-th object, so as to constrain the similarity of the m optimized features to the corresponding similarity around them, thereby increasing the diversity between face images of the same object.

[0059] To address the issue of the limited variety of sample images used in training image recognition models, this application proposes an image generation method. Figure 2 This is a flowchart of an image generation method according to an embodiment of this application, which includes the following steps S202 to S208.

[0060] It should be noted that the image generation method shown in steps S202 to S208 can be, but is not limited to, executed by an electronic device, wherein the electronic device can be, but is not limited to, as shown in the example below. Figure 1 The target terminal or server shown.

[0061] Step S202: Obtain the identity features of the i-th object among the K objects to obtain the i-th identity feature, where the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K;

[0062] Step S204: Obtain m initial features of the i-th object, where each of the m initial features is used to characterize a kind of object information of the i-th object, including identity information and attribute information, and m is a positive integer greater than or equal to 2;

[0063] The attribute information of the i-th object includes, but is not limited to, the attribute information of the face image in multiple dimensions such as illumination, pose, expression, and color saturation. For each initial feature of the i-th object, it includes its identity information and the attribute information in the above multiple dimensions, but the parameter values ​​corresponding to the attribute information in the multiple dimensions included in each initial feature are different.

[0064] For example, for three facial images of a person, each image contains attribute information in multiple dimensions such as lighting, pose, expression, and color saturation. However, the attribute information in multiple dimensions corresponding to each image is not the same. For example, the expression of the first facial image is smiling, the pose is upright, and the lighting parameter is the first value. The expression of the second facial image is facing north, the pose is walking, and the lighting parameter is the second value.

[0065] Step S206: Obtain m similarity thresholds for the i-th object;

[0066] Step S208: Based on the i-th identity feature and m similarity thresholds, perform multiple rounds of iterative adjustments on the m initial features to obtain m optimized features. Among these, the m optimized features have the smallest difference between the m similarities and the m similarity thresholds between the i-th identity feature and the m optimized features. Different optimized features represent different attribute information, and different optimized features are used to generate different images of the i-th object.

[0067] For ease of understanding, the above image generation method is explained using sample images of a face recognition model as examples in the embodiments of this application.

[0068] In the training process of facial recognition algorithms, the key to data generation lies in the following two points:

[0069] (1) Maintain consistency of intra-class data, that is, image data of the same identity in the training data should not be mixed with image data of other identities. If there is inconsistency in intra-class data, it will lead to a decrease in the accuracy of the face recognition model.

[0070] (2) Ensure that the training face data maintains the diversity of intra-class data, that is, data of the same identity have different poses, expressions, lighting, age and other attribute information that are unrelated to identity, increase the diversity of data and ensure the robustness of face recognition model.

[0071] In this application embodiment, a face feature sampling method based on hyperspherical uniform constraints is proposed. Starting from the basic assumption that the features of face recognition are distributed in hyperspace, the generated face data is unified with the constraints of the face recognition model.

[0072] In other words, the technical solution in this application embodiment adopts a two-stage sampling method. First, the identities in the feature sampling process are constrained to prevent overlap, thus ensuring the independence of inter-class identities (different identities). Second, based on the identity features of each identity, diversity sampling is performed on the intra-class features (initial features) of that identity. Based on the two-stage constrained sampling, the categorical diversity of the basic driving features and the richness of intra-class features are guaranteed, while the independence of inter-class identity features is also guaranteed, thereby affecting the generated face image to have the characteristics of non-overlapping inter-class features and intra-class diversity.

[0073] like Figure 3 As shown, the image generation method in this embodiment includes two parts: acquiring driving features and generating real face images (which can also be understood as real face data). The overall steps are as follows:

[0074] S302, Obtain inter-class identity features;

[0075] In this embodiment, feature vectors of identity features of different objects uniformly distributed in the hyperspherical space can be obtained by means of parameter optimization, but are not limited to. Assuming that the feature vector of each identity feature is 1×D in dimension, D = 512, and the number of objects is K, this means that the number of features to be sampled is K.

[0076] S304, Expanding intra-class features (initial features) based on inter-class identity features;

[0077] This step is used to obtain diverse intra-class facial features corresponding to each object in an optimized manner. The method of obtaining diverse intra-class features will be described below with reference to specific embodiments.

[0078] S306, Training the generative model;

[0079] In the embodiments of this application, the generative model may be, but is not limited to, a diffusion model. The principle of the diffusion model is to learn the information attenuation caused by noise, and then use the learned pattern to generate an image. The model gradually obfuscates the data by adding Gaussian noise, and then learns to reverse the noise process to reconstruct the sample.

[0080] S308 drives the generative model based on the features obtained from sampling.

[0081] Assuming that each of the m initial features of the i-th object comprises a set of features, and that the set of features exists in matrix form, the above methods for optimizing multiple initial features of the i-th object also include:

[0082] Obtain the identity features of the i-th object among K objects to get the i-th identity feature, where K is a positive integer greater than or equal to 2 and i is a positive integer less than or equal to K;

[0083] Obtain the initial features of the i-th object, and get the i-th initial features, where the i-th initial features include the initial feature values ​​of the i-th object in m×D dimensions, where m and D are positive integers greater than or equal to 2;

[0084] Obtain the similarity parameter of the i-th object, and get the i-th similarity parameter, where the i-th similarity parameter includes the similarity threshold in m×1 dimensions;

[0085] Based on the i-th identity feature and the i-th similarity parameter, the i-th initial feature is adjusted to obtain the i-th optimized feature. The i-th optimized feature includes the optimized feature values ​​of the i-th object in m×D dimensions. In the i-th optimized feature, the feature values ​​in the same column and the feature values ​​in different rows are used to generate different images of the i-th object.

[0086] like Figure 9 As shown, assuming there are 4 objects in total, each row in the inter-class identity feature X represents the identity feature of an object. For example, (x 11 ,x 12 ,x 13 ,x 14 ,x 15 ,x 16 ,x 17 ,x 18 ,) represents the identity characteristics of the first object.

[0087] Based on the identity features of the i-th object, the initial features of the i-th object are obtained. It is assumed that the i-th initial feature of the i-th object includes multiple attribute information of a face image. The attribute information includes, but is not limited to, pose, expression, lighting, age, etc. Each type of attribute information is represented by a 1×D dimensional feature vector (which can also be understood as a sub-feature of the i-th initial feature). When there are m types of attribute information, the i-th initial feature can be represented by, but is not limited to, an in-class feature matrix of dimension m×D.

[0088] The initial features of the i-th object and the initial features of the j-th object both include the aforementioned multiple attribute information. In other words, the parameter values ​​corresponding to the multiple attribute information contained in different images of the same person are different. For example, for two different images of the same person, both contain multiple attribute information such as lighting, color saturation, expression, and pose, but the values ​​of each parameter such as lighting parameter, expression parameter, and pose parameter are different in the two different images.

[0089] Based on the i-th identity feature and the i-th similarity parameter, the i-th initial feature is adjusted to obtain the i-th optimized feature with the same dimension of m×D. In the i-th optimized feature, the feature values ​​in the same column are different, and the feature values ​​in different rows are used to generate different images of the i-th object.

[0090] For example, assuming m=4, the first row of the i-th optimized feature represents the first intra-class sub-feature of the i-th object, and the second row represents the second intra-class sub-feature of the i-th object. The first intra-class sub-feature and the second intra-class sub-feature can be used to generate different images of the i-th object.

[0091] By iteratively adjusting the m initial features of the i-th object based on its identity features and m similarity thresholds, m optimized features are obtained. Specifically, the similarity between the m initial features and the i-th identity features is constrained to minimize the difference between the similarity to the m similarity thresholds, thus ensuring that the similarity of the adjusted m optimized features is constrained to around the target similarity. This allows multiple images of the i-th object to be generated using the m optimized features, increasing the diversity of sample images for the same object and improving the quality of sample images used to train the image recognition model.

[0092] As an optional example, the above method for obtaining m similarity thresholds for the i-th object includes:

[0093] Sample m similarity thresholds within the preset range of similarity threshold values.

[0094] Based on the number of features to be sampled m and the upper and lower bound thresholds t1 and t2 of intra-class similarity sampling, m similarity thresholds are sampled, where the upper and lower bound thresholds are set to (0.8, 0.9). A set of values ​​is sampled from (0.8, 0.9), and the i-th similarity parameter (which can also be understood as the similarity vector of the i-th object) is determined based on the set of values.

[0095] The intra-class similarity vector v, composed of m similarity thresholds, serves to constrain the diverse intra-class features that only the multiple initial features scattered around the i-th identity feature belong to the i-th object.

[0096] In other words, the role of the intra-class similarity vector v is to constrain the intra-class features scattered around the i-th identity feature to be the optimized features of the i-th object.

[0097] As an alternative implementation, the m similarity thresholds for the i-th object mentioned above include:

[0098] Select m similarity thresholds that conform to a Gaussian distribution.

[0099] Selecting m similarity thresholds that conform to a Gaussian distribution can, but is not limited to, help the process of iteratively adjusting the m initial features to converge more closely, thus improving the efficiency of the iterative adjustment.

[0100] Obviously, the Gaussian distribution is just one example and is not limited to it. For example, it could be a set of similarity thresholds for other distribution states.

[0101] In addition to sampling m similarity thresholds from a preset range of values ​​and selecting m similarity thresholds that conform to a Gaussian distribution, the m similarity thresholds for the i-th object can also be obtained in the following way: sample m similarity thresholds that conform to a Gaussian distribution within a preset range of similarity threshold values.

[0102] As an optional example, the m similarities between the above m optimized features and the i-th identity feature have the smallest differences with the m similarity thresholds, including:

[0103] Given m similarities, including the similarity between each of the m optimized features and the i-th identity feature, the average of the m differences is minimized. Here, the m differences include the differences between the similarity to the j-th optimized feature among the m similarities and the similarity thresholds corresponding to the j-th optimized feature among the m similarity thresholds, where j is a positive integer less than or equal to m. There is a one-to-one correspondence between the m similarities and the m optimized features, and a one-to-one correspondence between the m similarity thresholds and the m optimized features. Or...

[0104] The mean squared error of the m differences is minimized when the m similarities include the similarity between each of the m optimized features and the i-th identity feature.

[0105] Given that the initial features of the i-th object exist in the form of an in-class feature matrix, the following will first combine... Figure 6 The overall process of iterative processing of the intra-class feature matrix is ​​briefly described below.

[0106] In other words, the steps to determine the feature matrix of the i-th target class corresponding to the i-th object are as follows:

[0107] S602, Obtain the identity characteristics of each object;

[0108] The input to this module is the identity feature vector of dimension 1×D for each row of the target inter-class identity feature matrix determined in the above embodiments (which can also be understood as the class center feature vector w corresponding to each identity). i ), where i is the identity number obtained from the sampling.

[0109] S604, Obtaining the intra-class feature matrix;

[0110] Assuming the sampled identity number is i, then the i-th identity feature vector with dimension 1×D will be determined from the inter-class identity feature matrix. Based on the number m of features to be sampled, a matrix Y with dimension m×D will be obtained. i , where matrix Y i Each row represents an in-class feature of the i-th identity.

[0111] In other words, based on the pre-set number of features to be sampled m and the sampled i-th identity feature (which can also be understood as the i-th initial identity feature), the initialized i-th intra-class feature matrix Y is obtained. i .

[0112] S606, Determine the intra-class similarity vector (similarity parameter);

[0113] Based on the number of features to be sampled m and the upper and lower bound thresholds t1 and t2 for intra-class similarity sampling, the threshold is usually set to (0.8, 0.9). For the specific method of obtaining the threshold, please refer to the description of obtaining m similarity thresholds in the above section. It will not be repeated here.

[0114] It should be noted that, if the i-th initial identity feature is regarded as the class center feature of the i-th identity, and the class center feature of the i-th identity is regarded as a center point A in the hyperspherical space, and the intra-class similarity vector v is regarded as a point B in the hyperspherical space, then the length of the line connecting the center point A and the point B is used as the radius, and the feature located in the part of the spherical space between the center point A and the point B in the hyperspherical space is regarded as the intra-class feature of the i-th identity.

[0115] In other words, the role of the intra-class similarity vector v is to constrain that only features scattered around the class center feature of the i-th identity belong to the diverse intra-class features of that identity.

[0116] S608, normalizes the initialized intra-class feature matrix;

[0117] By normalizing the initialized intra-class feature matrix, the i-th normalized intra-class feature matrix is ​​obtained.

[0118] S610, Calculate the value of the intra-class feature similarity loss function based on the i-th normalized intra-class feature matrix, the i-th identity feature, and the intra-class similarity vector;

[0119] S612, determine whether the number of iteration steps is satisfied;

[0120] If the condition is met, proceed to step S614; otherwise, proceed to step S616.

[0121] S614, the i-th normalized intra-class feature matrix that satisfies the iteration step number is determined as the i-th target intra-class feature matrix;

[0122] Alternatively, training can be stopped when the value of the second loss function satisfies the second convergence condition, and the i-th normalized intra-class feature matrix at the time of training stoppage can be determined as the i-th target intra-class feature matrix.

[0123] S616, Optimization of the loss objective function;

[0124] In this embodiment, a gradient descent-based approach (such as stochastic gradient descent, stochastic gradient descent with kinetic terms, adam, adagald) is used to train and optimize the network structure corresponding to the normalized intra-class feature matrix.

[0125] During training, steps S608 and S610 are repeated until the training results meet the training termination condition. The termination condition is usually set to the number of iterations meeting a set value, or the loss calculated by the similarity loss function (second loss function) being less than a set value, which completes the training of the model or network structure.

[0126] By using the above method to determine the intra-class feature matrix, more refined intra-class features of each object can be sampled for each object in the feature space, thereby expanding the feature space for each identity and further improving the diversity of face images of people with the same identity.

[0127] As another optional embodiment, when the i-th similarity parameter is composed of m similarity thresholds and each initial feature exists in matrix form, the i-th initial feature can be iteratively adjusted in multiple rounds based on the i-th identity feature and the i-th similarity parameter to obtain the i-th optimized feature. In the multiple rounds of iterative adjustment, the difference between the similarity between the i-th optimized feature and the i-th identity feature in the m×1 dimension and the similarity thresholds in the m×1 dimension included in the i-th similarity parameter is minimized, or the mean squared error of the difference is minimized.

[0128] Among them, the i-th initial feature can be adjusted through multiple rounds of iteration using, but is not limited to, the following formula (1):

[0129]

[0130] i represents the i-th object among K objects, j represents the initial feature of the i-th object in the j-th dimension, and w i Let v represent the i-th identity feature. i Y represents the i-th similarity parameter. ij The dimension is m×D, w i The dimension is 1×D, v i The dimension is m×1, where m represents that the i-th optimization feature includes m sub-optimization features in m dimensions.

[0131] As an optional example, based on the i-th identity feature and m similarity thresholds, the above process iteratively adjusts the m initial features to obtain m optimized features, including:

[0132] The m initial features are adjusted in the p-th iteration using the following steps, where p is a positive integer:

[0133] Obtain the similarity between each of the m features adjusted in the (p-1)th round of iterations and the i-th identity feature, and obtain the m similarities in the p-th round. When p equals 1, the m features adjusted in the (p-1)th round of iterations are the m initial features.

[0134] Based on the m similarities and m similarity thresholds in round p, the m differences in round p are determined. The m differences in round p include the difference between the j-th similarity among the m similarities in round p and the j-th similarity threshold among the m similarity thresholds, where j is a positive integer less than or equal to m. The m similarities in round p have a one-to-one correspondence with the m features after iterative adjustment in round p-1, and the m similarity thresholds have a one-to-one correspondence with the m features after iterative adjustment in round p-1.

[0135] Based on the m differences in the p-th round, determine the similarity loss value for the p-th round;

[0136] If the similarity loss value does not meet the preset first termination adjustment condition, the m features after the p-1th iteration adjustment are adjusted to obtain the m features after the pth iteration adjustment.

[0137] If the similarity loss value meets the first termination adjustment condition, the m features adjusted in the (p-1)th iteration are determined as the m optimized features.

[0138] Assuming K=4, m=2, D=8, using the above formula (1), calculate the difference e1 between the similarity between the i-th optimized feature and the i-th identity feature in the j=1 dimension and the similarity threshold in the m×1 dimension of the i-th similarity parameter, and the difference e2 between the similarity between the i-th optimized feature and the i-th identity feature in the j=2 dimension and the similarity threshold in the m×1 dimension of the i-th similarity parameter.

[0139] Through the above multiple rounds of iterative adjustments, the difference between the similarity between the i-th optimized feature and the i-th identity feature in the m×1 dimensions and the similarity threshold in the m×1 dimensions of the i-th similarity parameter can be minimized.

[0140] Specifically, the difference between the similarity between the i-th optimized feature and the i-th identity feature in the m×1 dimensions and the similarity threshold in the m×1 dimensions included in the i-th similarity parameter is the smallest, including:

[0141] When the similarity between the i-th optimized feature and the i-th identity feature in the m×1 dimensions includes m similarities and the similarity thresholds in the m×1 dimensions include m similarity thresholds, the average of the m differences is minimized. Here, the m differences include the differences between the similarity in the j-th dimension among the m similarities and the similarity thresholds in the j-th dimension among the m similarities, where j is a positive integer less than or equal to m.

[0142] For example, when the dimension of the i-th optimized feature is m×D(2×8), calculate the first similarity between the first sub-feature (dimension 1×8) corresponding to the first row of the i-th optimized feature and the i-th identity feature, and then determine the difference e1 between the first similarity and the m=2 similarity thresholds in the i-th similarity parameter; calculate the second similarity between the second sub-feature (dimension 1×8) corresponding to the second row of the i-th optimized feature and the i-th identity feature, and then determine the difference e2 between the second similarity and the m=2 similarity thresholds in the i-th similarity parameter.

[0143] Specifically, in the current round of iterative adjustment, by utilizing the difference between the similarity between the i-th optimized feature and the i-th identity feature in the m×1 dimension and the similarity threshold in the m×1 dimension included in the i-th similarity parameter, the i-th initial feature is made to iterate in the direction of minimizing the above difference.

[0144] As an optional example, the above method for determining the similarity loss value for round p based on the m differences in round p includes:

[0145] The similarity loss value in round p is determined to be equal to the average of the m differences in round p; or

[0146] The similarity loss value in round p is determined to be equal to the mean squared error of the m differences in round p.

[0147] As can be seen from the above formula (1), the above-mentioned iteration of the i-th initial feature in the direction of minimizing the aforementioned difference can also be understood as minimizing the average value of the above m=2 differences.

[0148] In other words, the ideal state of the above iterative adjustment is that the difference between the similarity between each sub-feature in the i-th optimized feature and the i-th identity feature and the i-th similarity parameter is 0. That is, the similarity between each sub-feature in the i-th optimized feature and the i-th identity feature is fitted to the target similarity (e.g., m similarity thresholds in the i-th similarity parameter). The purpose of optimizing the i-th initial feature using the i-th similarity parameter is to constrain each feature within the i-th object class to the corresponding similarity.

[0149] As an optional implementation, the above method adjusts the m features after the (p-1)th iteration to obtain the m features after the pth iteration, including:

[0150] Reduce at least some features among the m features adjusted in the (p-1)th iteration to obtain the m features adjusted in the pth iteration; or

[0151] Decrease the first part of the m features after the (p-1)th iteration adjustment, and add the second part of the m features after the (p-1)th iteration adjustment to obtain the m features after the pth iteration adjustment, where the first part of the features and the second part of the features are different.

[0152] With m=2 and D=8, the i-th initial feature is a 2×8 intra-class feature matrix. The first row of the intra-class feature matrix represents the first sub-feature of the i-th object, and the second row represents the second sub-feature of the i-th object.

[0153] Suppose the first action of the (p-1)th feature is [v] 11 ,v 12 ,v 13 ,v 14 ,v 15 ,v 16 ,v 17 ,v 18 The second line [v] 21 ,v 22 ,v 23 ,v 24 ,v 25 ,v 26 ,v 27 ,v28 If the similarity loss does not meet the constraint's termination adjustment condition, then reduce [v]. 11 ,v 12 ,v 13 ,v 14 ,v 15 ,v 16 ,v 17 ,v 18 ] and [v 21 ,v 22 ,v 23 ,v 24 ,v 25 ,v 26 ,v 27 ,v 28 At least some of the feature values ​​in ] are used to obtain the adjusted p-th feature.

[0154] For example, [1,1,1,1,0,1,1,1] and [1,0,1,1,0,1,1,1] are adjusted to [0,0,1,1,0,1,1,1] and [0,0,1,1,0,1,1,0] respectively.

[0155] Or decrease [v] 11 ,v 12 ,v 13 ,v 14 ,v 15 ,v 16 ,v 17 ,v 18 ] and [v 21 ,v 22 ,v 23 ,v 24 ,v 25 ,v 26 ,v 27 ,v 28 The first part of the feature value in ] is increased, and the second part of the feature value in both is increased. For example, [1,1,1,1,0,1,1,1] and [1,0,1,1,0,1,1,1] are adjusted to [0,0,1,1,1,1,1,1] and [0,1,1,1,0,1,0,0] respectively.

[0156] In other words, during the optimization of the i-th initial feature, the feature values ​​of each row of sub-features in the i-th feature are adjusted, and the i-th similarity parameter is used to make the similarity between the adjusted i-th optimized feature and the i-th identity feature approach the target similarity.

[0157] As an optional example, the above method of obtaining the identity features of the i-th object out of K objects, to obtain the i-th identity feature, includes:

[0158] Select the identity feature of the i-th object from the K identity features to obtain the i-th identity feature. The K identity features include the identity features of each of the K objects, and each of the K identity features is used to represent the identity information of the corresponding object among the K objects; or

[0159] Select the identity feature of the i-th object from the K identity features, and normalize the identity feature of the i-th object to obtain the i-th identity feature.

[0160] When considering the K identity features of K objects as an inter-class identity feature matrix, it is possible, but not limited to, through... Figure 5 The flowchart shown above obtains the inter-class identity feature matrix. The specific process is as follows:

[0161] S502, Initialize the inter-class identity feature matrix (also known as the inter-class feature matrix);

[0162] The input for this step is an identity feature vector of dimension 1×D. Assuming that the number of features to be sampled is K, the initial inter-class feature matrix X of dimension K×D is obtained through initialization, where each row in X represents the central feature of each identity.

[0163] S504, normalize the initial inter-class feature matrix to obtain the normalized inter-class identity feature matrix;

[0164] In this process, each identity feature in the initial inter-class feature matrix is ​​normalized to obtain the identity features located in the hyperspherical space.

[0165] S506, Calculate the value of the inter-class similarity loss function;

[0166] The input to this module is a normalized inter-class identity feature matrix. It calculates the inner product of the normalized inter-class identity feature matrix, constrains the interval between inter-class identity features (the identity of each object), and calculates the value of the inter-class similarity loss function. The iterative processing of initializing the inter-class feature matrix will be described in detail below with reference to specific embodiments.

[0167] S508 trains and optimizes the network structure corresponding to the normalized inter-class identity feature matrix based on gradient descent methods (such as stochastic gradient descent, stochastic gradient descent with kinetic terms, adam, adagard).

[0168] During the training process, steps S502 to S506 are repeated until the training result meets the training termination condition. The condition for terminating model training is generally set to meet the set value in this iteration, or the loss calculated by the similarity loss function is less than the set value, which completes the training of the model.

[0169] S510, if the value of the first loss function satisfies the first convergence condition, stop training, complete the sampling of inter-class features, and determine the normalized inter-class identity feature matrix at the end of training as the target inter-class identity feature matrix.

[0170] As another alternative example, the above method of obtaining the identity characteristics of the i-th object among K objects includes:

[0171] Select the identity feature of the i-th object from the K identity features to obtain the i-th identity feature. The K identity features include the identity features of each of the K objects, and the i-th identity feature includes the feature values ​​of the i-th object in 1×D dimensions; or

[0172] Select the identity feature of the i-th object from the K identity features, and normalize the feature values ​​included in the identity feature of the i-th object to obtain the i-th identity feature. Here, the K identity features include the identity features of each of the K objects, and the i-th identity feature includes the feature values ​​of the i-th object in 1×D dimensions.

[0173] refer to Figure 9 (New addition) Figure 9 Let K = 4, D = 8, and X represent the inter-class identity feature matrix, where each row of matrix X represents the identity feature of an object.

[0174] The methods for obtaining the i-th identity feature of the i-th object include, but are not limited to, directly selecting the i-th object's identity feature from the four identity features as the i-th identity feature; or first selecting one identity feature from the four identity features, and then normalizing the selected identity feature to obtain the i-th identity feature.

[0175] As an optional example, the above method also includes:

[0176] Obtain the initial identity features of each of the K objects to obtain the K initial identity features;

[0177] The K initial identity features are iteratively adjusted in multiple rounds to obtain the K identity features.

[0178] In Figure 9 Given that the four rows in matrix X are determined as the initial identity features of four objects, the adjusted identity features are determined as the identity features of the four objects through multiple rounds of iterative adjustment of the initial identity features of the four objects.

[0179] As an optional example, the above method obtains the initial identity features of each of the K objects, resulting in K initial identity features, including:

[0180] Sampling initial identity features of K objects from a set of M initial identity features yields K initial identity features. The M initial identity features include the initial identity features of each of the M objects, where M is a positive integer greater than K.

[0181] Suppose that there are 10 initial identity features of M=10 objects distributed in the hyperspherical space. It is possible, but not limited to, to sample the initial identity features of 4 objects from the 10 initial identity features to obtain the initial identity features of the above 4 objects.

[0182] Optionally, the above process involves multiple rounds of iterative adjustments to the K initial identity features to obtain K identity features, including:

[0183] Multiple rounds of iterative adjustments are made to the K initial identity features to obtain K identity features. Among these multiple rounds of iterative adjustments, the loss value determined based on the K identity features is the minimum.

[0184] The loss value determined based on the K identity features can be achieved, but is not limited to, through the following formulas (2) and (3):

[0185]

[0186] M = XX T -2I (3)

[0187] Where i and j represent two different objects, I is the identity matrix, and M can be ensured by formula (3). ij The calculation yields the similarity between the identity features of each object and other objects. That is, the similarity between each object and its own identity features can be filtered out by formula (3).

[0188] For example, assuming there are 4 objects, M can be filtered out using formula (3). 11 M 22 M 33 M 44 Ensure that the calculated similarity is the similarity between the identity features of two different objects, for example, M 12 M 13 M 23 M 12 .

[0189] For example, suppose there are 10 different identities, including 5 adults and 5 minors. By taking the maximum value in formula (2), the facial features obtained by sampling using the identity features of the first identity (adult) are the facial features of one of the adults that are closest to it.

[0190] By taking the maximum value of the similarity between the identity features of each object and the identity features of other objects in the above formula (2), the facial features of the sampled different objects are relatively similar, but can be distinguished by the identity features of each object.

[0191] That is, by using formula (2), we can obtain the inter-class identity features of a group of objects with similar identity features and obvious identity differences. By using this stable constraint method to generate and sample features, we can improve the efficiency and accuracy of feature sampling.

[0192] As an optional example, the above iterative adjustment of the K initial identity features yields K identity features, including:

[0193] The K initial identity features are adjusted in the qth iteration using the following steps, where q is a positive integer:

[0194] Based on the K features adjusted in the (q-1)th iteration, the loss value in the qth iteration is determined, where when q equals 1, the K features adjusted in the (q-1)th iteration include K initial identity features;

[0195] If the loss value in round q does not meet the preset second termination adjustment condition, the K features after the iteration adjustment in round q-1 are adjusted to obtain the K features after the iteration adjustment in round q.

[0196] If the loss value in round q satisfies the second termination adjustment condition, the K features adjusted in round q-1 are determined as K identity features.

[0197] When the K features adjusted based on the (q-1)th iteration are determined as the (q-1)th matrix, the loss value matrix for the qth round can be determined based on, but is not limited to, the (q-1)th matrix. When q equals 1, the (q-1)th matrix includes K×D initial feature values ​​composed of K initial identity features. The loss value for the qth round is determined based on the loss value matrix for the qth round. If the loss value for the qth round does not meet the preset second termination adjustment condition, the (q-1)th matrix is ​​adjusted to obtain the qth matrix after the qth iteration adjustment. If the loss value for the qth round meets the second termination adjustment condition, the (q-1)th matrix is ​​determined as the K identity features.

[0198] like Figure 9 As shown, assuming the (q-1)th matrix is ​​X, each row of matrix X contains a 1×8 vector representing the initial identity features of an object. The loss matrix for the qth round is determined by transposing matrix X in formula (3). Furthermore, the similarity M is considered during the calculation of the loss value for the qth round. 11 M 22 M33 M 44 It is not included in the calculation.

[0199] As an optional example, the above method, based on the K features adjusted in the (q-1)th iteration, determines the loss value for the qth round, including:

[0200] Multiply the (q-1)th matrix with the transpose of the (q-1)th matrix to obtain the first matrix, where the (q-1)th matrix includes a matrix composed of K features after the (q-1)th round of iterative adjustment;

[0201] Multiply the identity matrix by the preset coefficients to obtain the second matrix;

[0202] Subtracting the second matrix from the first matrix yields the loss matrix for the qth round.

[0203] Determine the loss value for round q based on the loss value matrix for round q.

[0204] The first matrix can be referenced from the newly added one. Figure 9 The second matrix is ​​obtained by formula (3) above. The loss matrix of the qth round is obtained by subtracting the eigenvalues ​​at the corresponding positions of the first matrix and the second matrix.

[0205] As an optional example, the above method of determining the loss value in round q based on the loss value matrix in round q includes:

[0206] Obtain the largest eigenvalue in each row of the loss matrix in the q-th round, and obtain K eigenvalues;

[0207] The loss value in the qth round is determined as the average or mean squared error of the K features.

[0208] Specifically, in determining the loss value of the qth round using the above formula (2), assuming K = 4, when i = 1, the similarity with the largest value in the first row of the loss value matrix is ​​determined (for example, M). 13 When i = 2, determine the similarity with the largest value in the second row of the loss value matrix (e.g., M). 23 When i = 3, determine the similarity with the largest value in the third row of the loss value matrix (e.g., M). 34 When i = 4, determine the similarity with the largest value in the fourth row of the loss value matrix (e.g., M). 41 ).

[0209] Then the loss value in round q is equal to: for M 13 M 23 M 34 M 41Summing is performed, and the summation value is divided by K=4. If the loss value in round q does not meet the second termination adjustment condition, the feature values ​​contained in at least some rows of X are adjusted.

[0210] From the perspective of the inner product operation of matrix X and the transpose of matrix X, the similarity M 13 It equals the identity feature X1 of the first object and the identity feature X of the third object. T The inner product of 3, therefore, in the q-th iteration adjustment process, it is necessary to adjust the identity feature X1 and the identity feature X3 of the third object; similarly, the similarity M 23 It equals the identity feature X2 of the second object and the identity feature X of the third object. T The inner product of 3, therefore, in the q-th iteration adjustment process, it is necessary to adjust the identity feature X2 and the identity feature X3 of the third object; and so on, to adjust the identity features associated with the largest similarity value.

[0211] As an optional example, the above adjustment of the K features after the (q-1)th iteration yields the K features after the qth iteration, including:

[0212] Reduce at least some of the K features after the (q-1)th iteration adjustment to obtain the K features after the qth iteration adjustment; or

[0213] Decrease the third part of the K features after the (q-1)th iteration adjustment, and add the fourth part of the K features after the (q-1)th iteration adjustment to obtain the K features after the qth iteration adjustment, where the third part and the fourth part are different.

[0214] Specifically, adjustments may include, but are not limited to, certain feature values ​​contained in the identity characteristics of each object, or feature values ​​contained in the identity characteristics of a subset of objects. For example, adjustments may include, but are not limited to, (x) 11 ,x 12 ,x 13 ,x 14 ,x 15 ,x 16 ,x 17 ,x 18 At least some of the feature values ​​in ); or adjust at least some of the feature values ​​in the identity features X1, X2, X3, X4 of each object, etc.

[0215] By using the above method, we can obtain inter-class identity features corresponding to a group of objects with similar identity characteristics and obvious identity differences. Using this stable constraint method to generate and sample features can improve the efficiency and accuracy of feature sampling.

[0216] As an optional example, after iteratively adjusting m initial features based on the i-th identity feature and m similarity thresholds to obtain m optimized features, the above method further includes:

[0217] Based on each of the m optimized features and the initial noisy image, generate m candidate images for the i-th object.

[0218] Each of the m candidate images is generated based on the corresponding optimized features from the m optimized features and the initial noisy image.

[0219] Optionally, the above process of generating m candidate images for the i-th object based on each of the m optimized features and the initial noisy image includes:

[0220] Generate one candidate image from m candidate images using the following steps:

[0221] A candidate image is obtained by performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image.

[0222] After determining the i-th identity feature of the i-th object, it is possible, but not limited to, to use features such as... Figure 4 The flowchart shown generates a candidate image carrying an optimized feature of the i-th object.

[0223] S402, extract the i-th identity feature of the i-th object through the face feature extraction unit module;

[0224] The function of this module is to randomly generate a feature vector y representing a face identity in spherical space. ij Where i represents the i-th identity among multiple identities, and j represents the j-th intra-class feature corresponding to the i-th identity. Assuming that the dimension of the identity feature of each object is 1×512, then the dimension of the j-th initial feature corresponding to the i-th identity is also 1×512.

[0225] S404, randomly acquire the initial noisy image;

[0226] In this embodiment, it is assumed that the size of the noisy image is 512×512.

[0227] S406, the initial noisy image is denoised using the extracted i-th identity feature in the diffusion network, resulting in a first denoised face image with a size still 512×512. The training process of the diffusion network will be described in detail below with reference to specific embodiments.

[0228] Specifically, the first denoised face image output by the diffusion network can be determined using the following formula (4):

[0229]

[0230] The output of the diffusion network is p(x) t |y), the diffusion network is represented as x t The output of the diffusion model at iteration step t. The noise coefficient is preset for the training phase, and y represents the feature vector of the face identity (which can also be understood as the j-th intra-class feature vector of the i-th identity).

[0231] S408, In the identity constraint network, the first denoised face image is denoised again using the i-th identity feature to obtain the second denoised face image carrying the i-th identity feature.

[0232] The size of the second denoised face image remains 512×512. The specific implementation process of generating the second denoised face image carrying the i-th identity feature using the identity constraint network will be described in detail in the following embodiments.

[0233] S410, the first denoised face image and the second denoised face image are fused by the image adjustment module to obtain the target face image carrying the target identity features. The processing process of the diffusion network will be described in detail below with reference to specific embodiments.

[0234] By generating an image adjustment module, the noise signals generated by the diffusion network and the identity constraint network are fused and processed, thereby denoising the randomly initialized noisy image. This can also be understood as summing the outputs of formula (4) and the following formula (5) to obtain the output of the current iteration.

[0235] As an optional implementation, the above method involves performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image to obtain a candidate image, including:

[0236] The following steps are used to perform the t-th iteration processing on one of the m optimized features and the initial noisy image, where t is a positive integer:

[0237] The noise image after the (t-1)th iteration is denoised based on an optimized feature to obtain the first estimated noise image of the tth iteration. When t=1, the noise image after the (t-1)th iteration is the initial noise image.

[0238] Based on an optimization feature, the noise image after the (t-1)th iteration is constrained to obtain the second estimated noise image for the tth iteration.

[0239] Subtract the first estimated noise image from round t and the second estimated noise image from round t from the noise image after the (t-1)th iteration to obtain the noise image after the tth iteration.

[0240] If t reaches the preset number of iterations, the noisy image after the t-th iteration is determined as a candidate image.

[0241] The above iterative process can be understood as follows:

[0242] (1) Input the noise image after the (t-1)th iteration and an optimized feature into the diffusion network to obtain the first estimated noise image in the tth iteration, where the noise image after the (t-1)th iteration is the initial noise image when t=1;

[0243] (2) Input the noise image after the (t-1)th iteration and an optimized feature into the identity constraint network to obtain the second estimated noise image in the tth iteration;

[0244] (3) Subtract the first estimated noise image of round t and the second estimated noise image of round t from the noise image after the (t-1)th iteration to obtain the noise image after the tth iteration;

[0245] (4) When t reaches the preset number of iterations, the noise image after the tth iteration is determined as a candidate image.

[0246] For example, in the first iteration: assuming an initial noise N0 is input, by inputting the initial noise N0 into the diffusion network, the estimated noise n is obtained. 1-1 By inputting the initial noise N0 into the identity constraint network, the estimated noise n is obtained. 2-1 Subtract the estimated noise n from the initial noise N0 1-1 and n 2-1 The noise image N1 after the first round of iterations is obtained.

[0247] Second iteration: Input the noisy image N1 into the diffusion network to obtain the estimated noise n 1-2 Input the noisy image N1 into the identity constraint network to obtain the estimated noise n. 2-2 Then subtract the estimated noise n from the noise N1. 1-2 and n 2-2 The noise image N2 after the second iteration is obtained.

[0248] Similarly, after each round of iteration, it is determined whether the number of iterations has reached the preset threshold. If it has, the iteration process is stopped, and a face image in RGB space is generated based on the noise image from the previous round. The generated face image can then be identified as a candidate image for the i-th object.

[0249] The above constraint processing can be performed, but is not limited to, by using the following formula (5) to denoise the first estimated noise image of the input identity constraint network again to obtain the second estimated noise image:

[0250]

[0251] Where p(y|x) t ) represents the output of the identity constraint network, which constrains the generated image to have a specific identity. t Let κ be the coefficient of the identity-constrained network. t These are artificially set hyperparameters, usually related to κ in formula (7) during the training phase. s Keep it consistent, f φ For face recognition models, x t The output of the diffusion model at the current iteration step t. The derivative symbol is used to extract the identity features of the i-th object. The derivative of the calculation model with respect to the input is used to calculate the derivative of the face identity feature vector. T represents the transpose.

[0252] By multiplying the denoised face image obtained from the above differentiation with the feature vector representing the face identity, the selected i-th identity feature can be made more prominent in the second estimated noisy image, thereby increasing the difference between intra-class face data of the same object.

[0253] The above method can generate multiple candidate images for each object, and each candidate image carries identity features, which improves the diversity of training sample images for the same object.

[0254] In addition, before using target identity features in the diffusion network to denoise the initial noise image to generate an estimated noise image, the above method further includes: acquiring sample noise images; extracting features from the sample noise images to obtain sample noise image features; inputting the sample noise image features into the initialized diffusion network for training to obtain the value of the third loss function corresponding to the diffusion network; inputting the sample noise images and sample identity features into the initialized identity constraint network to obtain the value of the fourth loss function corresponding to the identity constraint network; determining the value of the target loss function based on the values ​​of the third and fourth loss functions, and training the initialized diffusion network and the initialized identity constraint network based on the value of the target loss function until the value of the target loss function satisfies the target convergence condition.

[0255] For details, please refer to... Figure 7 The flowchart of the training process for the generative model is shown below, with detailed steps as follows:

[0256] S702, prepare training data;

[0257] The training data preparation module reads the face training data during training, combines the read data into a batch, and sends it to the network unit for processing. The face image size is transformed to 256×256, and then the image pixel values ​​are normalized to the range [-1, 1]. Simultaneously, this module reads the feature information corresponding to the face identity of each image. This feature information is extracted from each face image by the face recognition model.

[0258] S704 extracts features from a face image through a face feature extraction network unit module to obtain identity features carrying face image identity information;

[0259] For example, extract the identity features of objects in the original face image, or the intra-class features corresponding to the i-th identity mentioned above.

[0260] S706, in the diffusion network, the randomly sampled initial noisy face image is diffused to denoise, and a face image that conforms to the distribution characteristics of the training data is obtained;

[0261] During training, the input to this module is the facial features extracted from the original face image. In each iteration of training, the number of diffusion steps t and the corresponding face image with added noise at step t are sampled. Diffusion networks typically have a UNET structure, which includes operations such as convolution computation, non-linear activation function (ReLU) computation, pooling computation, fully connected layers, and various transformer structures.

[0262] S708, Determine the value of the loss function for the diffusion network;

[0263] The target function value is calculated by using the loss function calculation unit module commonly used in diffusion networks, taking the size of the noisy face image obtained after passing through the diffusion model network unit module and the current noisy face image as input.

[0264] The loss function of the diffusion network can be, but is not limited to, the MSE function. The loss function of the diffusion network is as follows (6):

[0265]

[0266] Where x0 is the original face data, ∈ is noise randomly sampled from a Gaussian distribution, ∈ θ Here, θ represents the parameters of the diffusion model, and t represents the number of diffusion steps sampled in each training iteration. The value of t ranges from 0 to 1000. α1 is the noise coefficient, the value of which is freely determined during training and is usually between 0 and 1. ∈ represents noise randomly drawn from a standard Gaussian distribution, and c represents the input condition of the diffusion network. In this embodiment, c represents the identity features of a face.

[0267] S710, Calculation of the identity feature constraint loss function;

[0268] The identity features represented by the original face image, the identity features extracted from the noisy face image generated by the diffusion network, and the number of diffusion steps t used to train the current diffusion network are taken as inputs, and the value of the identity constraint loss function is calculated using the following formula (7):

[0269]

[0270] κ s =0.5·(1-1 / (1+exp(-t / T)) (8)

[0271] Among them, κ s The scaling factor calculated for this loss varies with the number of iterations t, as shown in equation (8), where T = 1000 is the hyperparameter, and y T f represents the identity features extracted from the original image. φ (·) represents a pre-trained face recognition model. The noisy face image generated by the diffusion model.

[0272] S712, Determine the value of the total constraint loss function based on the value of the loss function of the diffusion network and the value of the identity feature constraint loss function;

[0273] Specifically, the values ​​of the loss function of the diffusion network and the identity feature constraint loss function are fused using the following formula (9) to obtain the value of the total constraint loss function:

[0274]

[0275] S714, Determine whether the conditions for terminating model training are met based on the value of the total constraint loss function;

[0276] If the condition is met, proceed to step S718; otherwise, proceed to step S716.

[0277] S716 adjusts the structural parameters of the diffusion network.

[0278] In this embodiment, the entire network can be trained and optimized using gradient descent methods (e.g., stochastic gradient descent, stochastic gradient descent with momentum terms, adam, adamard).

[0279] During the training process, repeat steps S702 to S716 until the training result meets the training termination condition. The condition for terminating model training is usually that the number of iterations meets the set value, or the loss calculated by the total constraint objective function is less than the set value.

[0280] S718, training stopped, optimization of the diffusion network unit module completed;

[0281] Using the above method, a diffusion network constrained by facial identity features can be trained to obtain a diffusion network that can generate noisy images.

[0282] To more clearly understand the above image generation method, the technical solution of this application is described below from the application side perspective, specifically as follows: Figure 10 The flowchart shown:

[0283] Step S1002: Obtain m optimized features and an initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting the m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature is used to represent the identity information of the i-th object, where i is a positive integer. Each of the m initial features is used to represent one type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multi-round iterative adjustment, the difference between the m similarities and m similarity thresholds between the m optimized features and the i-th identity feature is minimized. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0284] Step S1004: Based on each of the m optimized features and the initial noisy image, generate m candidate images for the i-th object.

[0285] Each of the m candidate images is generated based on the corresponding optimized feature from the m optimized features and the initial noisy image.

[0286] like Figure 9 As shown, assuming there are 4 objects in total, each row in the inter-class identity feature X represents the identity feature of an object. For example, (x 11 ,x 12 ,x 13 ,x 14 ,x 15 ,x 16 ,x 17 ,x 18 ,) represents the identity characteristics of the first object.

[0287] Based on the identity features of the i-th object, the initial features of the i-th object are obtained. It is assumed that the i-th initial feature of the i-th object includes multiple attribute information of a face image. The attribute information includes, but is not limited to, pose, expression, lighting, age, etc. Each type of attribute information is represented by a 1×D dimensional feature vector (which can also be understood as a sub-feature of the i-th initial feature). When there are m types of attribute information, the i-th initial feature can be represented by, but is not limited to, an in-class feature matrix of dimension m×D.

[0288] The initial features of the i-th object and the initial features of the j-th object both include the aforementioned multiple attribute information. In other words, the parameter values ​​corresponding to the multiple attribute information contained in different images of the same person are different. For example, for two different images of the same person, both contain multiple attribute information such as lighting, color saturation, expression, and pose, but the lighting parameter, expression parameter, and pose parameter values ​​are different in the two different images.

[0289] Based on the i-th identity feature and the i-th similarity parameter, the i-th initial feature is adjusted to obtain the i-th optimized feature with the same dimension of m×D. In the i-th optimized feature, the feature values ​​in the same column are different, and the feature values ​​in different rows are used to generate different images of the i-th object.

[0290] For example, assuming m=4, the first row of the i-th optimized feature represents the first intra-class sub-feature of the i-th object, and the second row represents the second intra-class sub-feature of the i-th object. The first intra-class sub-feature and the second intra-class sub-feature can be used to generate different images of the i-th object.

[0291] By iteratively adjusting the m initial features of the i-th object based on its identity features and m similarity thresholds, m optimized features are obtained. Specifically, the similarity between the m initial features and the i-th identity features is constrained to minimize the difference between the similarity to the m similarity thresholds, thus ensuring that the similarity of the adjusted m optimized features is constrained to around the target similarity. This allows multiple images of the i-th object to be generated using the m optimized features, increasing the diversity of sample images for the same object and improving the quality of sample images used to train the image recognition model.

[0292] As an optional example, the above generates m candidate images for the i-th object based on each of the m optimized features and the initial noisy image, including:

[0293] Generate one candidate image from m candidate images using the following steps:

[0294] A candidate image is obtained by performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image.

[0295] In such Figure 8 In the specific embodiment shown, it is assumed that the initial noisy image includes a total of 4 images n1, n2, n3, and n4, and a total of 3 identity features w1, w2, and w3 of 3 objects. The 3 intra-class features of each object are obtained, and the 3 intra-class features of each object are optimized using the identity features of each object to obtain 3 optimized features corresponding to each of the 3 objects.

[0296] Assuming the methods used in the above embodiments for determining the inter-class identity feature matrix and the intra-class feature matrix are adopted, and the three optimized feature w corresponding to the first object are determined sequentially... 11 w 12 and w 13 The three optimized identity features w corresponding to the second object 21 w 22 and w 23 The three optimized identity features w corresponding to the third object 31 w 32 and w 33 For details, please refer to the following: Figure 8 As shown in (a).

[0297] Suppose we use the first optimization feature w corresponding to the first object. 11 and utilize Figure 4 The generated image adjustment module shown fuses the denoised signals obtained from the diffusion network and the identity constraint network, thereby denoising the initial noisy image n1 to obtain the image shown. Figure 8 The candidate face image P shown in (b) 11-1 Following the same method, candidate face images P can also be obtained. 11-2 P 11-3 and P 11-4 .

[0298] As can be seen, by using an optimized feature corresponding to an object and a set of initial noisy images, we can obtain a set of multiple different face images with the same identity. Therefore, based on multiple optimized features corresponding to different objects, we can generate multiple different face images with different identities.

[0299] In other words, through the technical solutions in the above embodiments, multiple face images with the same identity can be generated by utilizing the optimized feature constraints corresponding to different object groups. This not only clearly distinguishes the inter-class data of different identities, but also ensures the diversity of intra-class data of the same identity, thereby providing high-quality training data for the training of the face recognition model.

[0300] It should be noted that the multiple candidate face images of each of the above objects can be used, but are not limited to, for training a face recognition model. The process of training a face recognition model using multiple candidate face images of an object includes:

[0301] 1) Obtain training data;

[0302] For example, suppose we select a portion of candidate face images from a face database consisting of candidate face images corresponding to each of the above objects as training samples.

[0303] 2) Preprocess some candidate face images;

[0304] For example, this includes image normalization and noise reduction.

[0305] 3) Extract features from the preprocessed candidate face images;

[0306] 4) Using the extracted feature vectors as input, the initial face recognition model is trained using a classification algorithm;

[0307] 5) Model evaluation: Use the test dataset to evaluate the trained model, compare the test results with the actual results, and verify the model's accuracy and generalization ability.

[0308] 6) Model optimization: Based on the evaluation results, optimize and adjust the initial face recognition model;

[0309] For example, adjust the structural parameters of the classifier and train it using the above process until the preset convergence condition is met.

[0310] In practical applications, multiple optimized features of multiple objects obtained in the embodiments of this application are typically used to generate multiple training samples for multiple objects, thereby diversifying the sample images used in the training process of the face recognition model and improving the accuracy and generalization ability of the face recognition model.

[0311] As an optional example, the above process involves multiple rounds of iterative processing of one of the m optimized features and the initial noisy image to obtain a candidate image, including:

[0312] The following steps are used to perform the t-th iteration processing on one of the m optimized features and the initial noisy image, where t is a positive integer:

[0313] The noise image after the (t-1)th iteration is denoised based on an optimized feature to obtain the first estimated noise image of the tth iteration. When t=1, the noise image after the (t-1)th iteration is the initial noise image.

[0314] Based on an optimization feature, the noise image after the (t-1)th iteration is constrained to obtain the second estimated noise image for the tth iteration.

[0315] Subtract the first estimated noise image from round t and the second estimated noise image from round t from the noise image after the (t-1)th iteration to obtain the noise image after the tth iteration.

[0316] If t reaches the preset number of iterations, the noisy image after the t-th iteration is determined as a candidate image.

[0317] For example, in the first iteration: assuming an initial noise N0 is input, by inputting the initial noise N0 into the diffusion network, the estimated noise n is obtained. 1-1 By inputting the initial noise N0 into the identity constraint network, the estimated noise n is obtained. 2-1 Subtract the estimated noise n from the initial noise N0 1-1 and n 2-1 The noise image N1 after the first round of iterations is obtained.

[0318] Second iteration: Input the noisy image N1 into the diffusion network to obtain the estimated noise n 1-2 Input the noisy image N1 into the identity constraint network to obtain the estimated noise n. 2-2 Then subtract the estimated noise n from the noise N1. 1-2 and n 2-2 The noise image N2 after the second iteration is obtained. This process is repeated until the number of iterations reaches a preset threshold. If the threshold is reached, the iteration process stops, and a face image in RGB space is generated based on the noise image from the previous iteration. The generated face image can then be identified as a candidate image for the i-th object.

[0319] Through the above iterative processing, the denoised signal obtained by the diffusion model and identity constraint processing can be fused to denoise the initial noisy image and obtain a candidate image of an object.

[0320] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0321] According to another aspect of the embodiments of this application, as follows is also provided Figure 11 An image generation apparatus is shown, the apparatus comprising:

[0322] The first acquisition unit 1102 is used to acquire the identity features of the i-th object among K objects to obtain the i-th identity feature, wherein the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K.

[0323] The second acquisition unit 1104 is used to acquire m initial features of the i-th object, wherein each of the m initial features is used to characterize a kind of object information of the i-th object, the object information includes identity information and attribute information, and m is a positive integer greater than or equal to 2;

[0324] The third acquisition unit 1106 is used to acquire m similarity thresholds for the i-th object;

[0325] The first processing unit 1108 is used to perform multiple rounds of iterative adjustment on m initial features based on the i-th identity feature and m similarity thresholds to obtain m optimized features. In the multiple rounds of iterative adjustment, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0326] Optionally, the third acquisition unit 1106 mentioned above includes:

[0327] The first sampling module is used to sample m similarity thresholds within a preset range of similarity threshold values.

[0328] Optionally, the third acquisition unit 1106 mentioned above includes:

[0329] The first processing module is used to select m similarity thresholds that conform to a Gaussian distribution.

[0330] Optionally, the first processing unit 1108 includes:

[0331] The second processing module is used to minimize the average of the m differences when considering the similarity between each of the m optimized features and the i-th identity feature, where each of the m similarities includes the similarity between the m similarities and the i-th identity feature. The m differences include the differences between the similarity between the m similarities and the j-th optimized feature and the similarity thresholds between the m similarity thresholds and the j-th optimized feature, where j is a positive integer less than or equal to m. There is a one-to-one correspondence between the m similarities and the m optimized features, and a one-to-one correspondence between the m similarity thresholds and the m optimized features.

[0332] The mean squared error of the m differences is minimized when the m similarities include the similarity between each of the m optimized features and the i-th identity feature.

[0333] Optionally, the first processing unit 1108 includes:

[0334] The third processing module is used to perform the p-th iteration adjustment on the m initial features through the following steps, where p is a positive integer:

[0335] Obtain the similarity between each of the m features adjusted in the (p-1)th round of iterations and the i-th identity feature, and obtain the m similarities in the p-th round. When p equals 1, the m features adjusted in the (p-1)th round of iterations are the m initial features.

[0336] Based on the m similarities and m similarity thresholds in round p, the m differences in round p are determined. The m differences in round p include the difference between the j-th similarity among the m similarities in round p and the j-th similarity threshold among the m similarity thresholds, where j is a positive integer less than or equal to m. The m similarities in round p have a one-to-one correspondence with the m features after iterative adjustment in round p-1, and the m similarity thresholds have a one-to-one correspondence with the m features after iterative adjustment in round p-1.

[0337] Based on the m differences in the p-th round, determine the similarity loss value for the p-th round;

[0338] If the similarity loss value does not meet the preset first termination adjustment condition, the m features after the p-1th iteration adjustment are adjusted to obtain the m features after the pth iteration adjustment.

[0339] If the similarity loss value meets the first termination adjustment condition, the m features adjusted in the (p-1)th iteration are determined as the m optimized features.

[0340] Optionally, the first acquisition unit 1102 mentioned above includes:

[0341] The fourth processing module is used to select the identity feature of the i-th object from the K identity features to obtain the i-th identity feature. The K identity features include the identity features of each of the K objects, and each of the K identity features is used to represent the identity information of the corresponding object among the K objects; or

[0342] Select the identity feature of the i-th object from the K identity features, and normalize the identity feature of the i-th object to obtain the i-th identity feature.

[0343] Optionally, the above-mentioned device further includes:

[0344] The fifth acquisition unit is used to acquire the initial identity features of each of the K objects, thus obtaining K initial identity features;

[0345] The first adjustment unit is used to perform multiple rounds of iterative adjustments on the K initial identity features to obtain K identity features.

[0346] Optionally, the first adjustment unit mentioned above includes:

[0347] The fifth processing module is used to perform multiple rounds of iterative adjustments on the K initial identity features to obtain K identity features. Among these multiple rounds of iterative adjustments, the loss value determined based on the K identity features is the minimum.

[0348] Optionally, the first adjustment unit mentioned above includes:

[0349] The sixth processing module is used to perform the q-th iteration adjustment on the K initial identity features through the following steps, where q is a positive integer:

[0350] Based on the K features adjusted in the (q-1)th iteration, the loss value in the qth iteration is determined, where when q equals 1, the K features adjusted in the (q-1)th iteration include K initial identity features;

[0351] If the loss value in round q does not meet the preset second termination adjustment condition, the K features after the iteration adjustment in round q-1 are adjusted to obtain the K features after the iteration adjustment in round q.

[0352] If the loss value in round q satisfies the second termination adjustment condition, the K features adjusted in round q-1 are determined as K identity features.

[0353] Optionally, the sixth processing module mentioned above includes:

[0354] The first processing submodule is used to multiply the (q-1)th matrix with the transpose of the (q-1)th matrix to obtain the first matrix, wherein the (q-1)th matrix includes a matrix composed of K features after the (q-1)th round of iterative adjustment;

[0355] The second processing submodule is used to multiply the identity matrix with preset coefficients to obtain the second matrix;

[0356] The third processing submodule is used to subtract the second matrix from the first matrix to obtain the loss value matrix for the qth round;

[0357] The fourth processing submodule is used to determine the loss value of round q based on the loss value matrix of round q.

[0358] Optionally, the sixth processing module mentioned above includes:

[0359] The first acquisition submodule is used to obtain the largest eigenvalue in each row of the loss matrix in the qth round, and obtain K eigenvalues;

[0360] The fifth processing submodule is used to determine the loss value of the qth round as the average or mean squared error of the K features.

[0361] Optionally, the sixth processing module mentioned above includes:

[0362] The sixth processing submodule is used to reduce at least some features among the K features adjusted in the (q-1)th iteration to obtain the K features adjusted in the qth iteration; or

[0363] The seventh processing submodule is used to reduce the third part of the K features after the (q-1)th iteration adjustment and add the fourth part of the K features after the (q-1)th iteration adjustment to obtain the K features after the qth iteration adjustment, wherein the third part of the features and the fourth part of the features are different.

[0364] According to another aspect of the embodiments of this application, as follows is also provided Figure 12 An image generation apparatus is shown, the apparatus comprising:

[0365] The fourth acquisition unit 1202 is used to acquire m optimized features and an initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting the m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature is used to represent the identity information of the i-th object, where i is a positive integer. Each of the m initial features is used to represent one type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multi-round iterative adjustment, the difference between the m similarities and the m similarity thresholds between the m optimized features and the i-th identity feature is minimized. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0366] The second processing unit 1204 is used to generate m candidate images of the i-th object based on each of the m optimized features and the initial noise image.

[0367] Optionally, the second processing unit 1204 includes:

[0368] The seventh processing module is used to generate one candidate image from m candidate images through the following steps:

[0369] A candidate image is obtained by performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image.

[0370] Optionally, the seventh processing module mentioned above includes:

[0371] The eighth processing submodule is used to perform the t-th iteration processing on one of the m optimized features and the initial noisy image through the following steps, where t is a positive integer:

[0372] The noise image after the (t-1)th iteration is denoised based on an optimized feature to obtain the first estimated noise image of the tth iteration. When t=1, the noise image after the (t-1)th iteration is the initial noise image.

[0373] Based on an optimization feature, the noise image after the (t-1)th iteration is constrained to obtain the second estimated noise image for the tth iteration.

[0374] Subtract the first estimated noise image from round t and the second estimated noise image from round t from the noise image after the (t-1)th iteration to obtain the noise image after the tth iteration.

[0375] If t reaches the preset number of iterations, the noisy image after the t-th iteration is determined as a candidate image.

[0376] By applying the aforementioned device to iteratively adjust m initial features of the i-th object based on its identity features and m similarity thresholds, m optimized features are obtained. Specifically, the similarity between the m initial features and the i-th identity feature is constrained to minimize the difference between the similarity and the m similarity thresholds, thus ensuring that the similarity of the adjusted m optimized features is constrained to around the target similarity. This allows multiple images of the i-th object to be generated using the m optimized features, increasing the diversity of sample images for the same object and improving the quality of sample images used to train the image recognition model.

[0377] It should be noted that the embodiments of the image generation apparatus described here can refer to the embodiments of the image generation method described above, and will not be repeated here.

[0378] According to another aspect of the embodiments of this application, an electronic device for implementing the above-described image generation method is also provided, the electronic device being... Figure 1 The target terminal or server is shown. This embodiment uses the electronic device as an example to illustrate the concept. Figure 13 As shown, the electronic device includes a memory 1302 and a processor 1304. The memory 1302 stores a computer program, and the processor 1304 is configured to execute the steps of any of the above method embodiments through the computer program.

[0379] Optionally, in this embodiment, the aforementioned electronic device may be located in at least one of a plurality of network devices in a computer network.

[0380] Optionally, in this embodiment, the processor can be configured to perform the following steps via a computer program:

[0381] S11, obtain the identity features of the i-th object among K objects, and obtain the i-th identity feature, where the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K;

[0382] S12, obtain m initial features of the i-th object, where each of the m initial features is used to characterize a kind of object information of the i-th object, the object information includes identity information and attribute information, and m is a positive integer greater than or equal to 2;

[0383] S13, obtain the m similarity thresholds for the i-th object;

[0384] S14. Based on the i-th identity feature and m similarity thresholds, the m initial features are iteratively adjusted in multiple rounds to obtain m optimized features. Among them, in the multiple rounds of iterative adjustment, the difference between the m similarities and m similarity thresholds between the m optimized features and the i-th identity feature is the smallest. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0385] Alternatively, the processor described above can be configured to perform the following steps via a computer program:

[0386] S21, obtain m optimized features and initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting the m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature is used to represent the identity information of the i-th object, where i is a positive integer. Each of the m initial features is used to represent one type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multi-round iterative adjustment, the difference between the m similarities and m similarity thresholds between the m optimized features and the i-th identity feature is minimized. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0387] S22, based on each of the m optimized features and the initial noisy image, generate m candidate images for the i-th object.

[0388] Alternatively, as those skilled in the art will understand, Figure 13 The structure shown is for illustrative purposes only. Figure 13This does not limit the structure of the aforementioned electronic devices or electronic equipment. For example, electronic devices or electronic equipment may also include components that are more... Figure 13 The more or fewer components shown (such as network interfaces, etc.), or having the same Figure 13 The different configurations shown.

[0389] The memory 1302 can be used to store software programs and modules, such as the program instructions / modules corresponding to the image generation method and apparatus in this embodiment. The processor 1304 executes various functional applications and data processing by running the software programs and modules stored in the memory 1302, thereby realizing the image generation method described above. The memory 1302 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1302 may further include memory remotely located relative to the processor 1304, and these remote memories can be connected to the terminal via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Specifically, the memory 1302 may be used, but is not limited to, to store the i-th identity feature, the initial noisy image, and candidate images, etc. As an example, such as... Figure 13 As shown in (a) and (b), the memory 1302 may include, but is not limited to, the first acquisition unit 1102, the second acquisition unit 1104, the third acquisition unit 1106, and the first processing unit 1108 of the image generation apparatus, or it may include the fourth acquisition unit 1202 and the second processing unit 1204 of the image generation apparatus. Furthermore, it may include, but is not limited to, other module units of the image generation apparatus, which will not be elaborated upon in this example.

[0390] Optionally, the transmission device 1306 described above is used to receive or send data via a network. Specific examples of the network described above may include wired networks and wireless networks. In one example, the transmission device 1306 includes a Network Interface Controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In another example, the transmission device 1306 is a Radio Frequency (RF) module, used for wireless communication with the Internet.

[0391] In addition, the aforementioned electronic device also includes: a display 1308 for displaying the initial noise image and candidate image; and a connection bus 1310 for connecting the various module components in the aforementioned electronic device.

[0392] In other embodiments, the target terminal or server can be a node in a distributed system, which can be a blockchain system. This blockchain system is formed by connecting multiple nodes through network communication. The nodes can form a point-to-point network, and any type of computing device, such as a server or target terminal, can become a node in the blockchain system by joining this point-to-point network.

[0393] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the image generation method provided in various optional implementations of the above-described server verification processing, wherein the computer program is configured to execute the steps in any of the above-described method embodiments at runtime.

[0394] Optionally, in this embodiment, the computer-readable storage medium described above may be configured to store a computer program for performing the following steps:

[0395] S11, obtain the identity features of the i-th object among K objects, and obtain the i-th identity feature, where the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K;

[0396] S12, obtain m initial features of the i-th object, where each of the m initial features is used to characterize a kind of object information of the i-th object, the object information includes identity information and attribute information, and m is a positive integer greater than or equal to 2;

[0397] S13, obtain the m similarity thresholds for the i-th object;

[0398] S14. Based on the i-th identity feature and m similarity thresholds, the m initial features are iteratively adjusted in multiple rounds to obtain m optimized features. Among them, in the multiple rounds of iterative adjustment, the difference between the m similarities and m similarity thresholds between the m optimized features and the i-th identity feature is the smallest. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0399] Alternatively, the aforementioned computer-readable storage medium may also be configured to store a computer program for performing the following steps:

[0400] S21, obtain m optimized features and initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting the m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature is used to represent the identity information of the i-th object, where i is a positive integer. Each of the m initial features is used to represent one type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multi-round iterative adjustment, the difference between the m similarities and m similarity thresholds between the m optimized features and the i-th identity feature is minimized. Different optimized features in the m optimized features represent different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object.

[0401] S22, based on each of the m optimized features and the initial noisy image, generate m candidate images for the i-th object.

[0402] Optionally, in embodiments of this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0403] Optionally, in this embodiment, those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing the hardware related to the target terminal. The program can be stored in a computer-readable storage medium, which may include: flash drive, read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0404] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0405] If the integrated units in the above embodiments are implemented as software functional units and sold or used as independent products, they can be stored in the aforementioned computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause one or more computer devices (which may be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods of the various embodiments of this application.

[0406] In the above embodiments of this application, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0407] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces, or the indirect coupling or communication connection of units or modules may be electrical or other forms.

[0408] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0409] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0410] The above are merely preferred embodiments of this application. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this application, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. An image generation method, characterized in that, include: Obtain the identity features of the i-th object among K objects to obtain the i-th identity feature, wherein the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K; Obtain m initial features of the i-th object, wherein each of the m initial features is used to characterize a kind of object information of the i-th object, the object information including identity information and attribute information, and m is a positive integer greater than or equal to 2; Obtain the m similarity thresholds for the i-th object; Based on the i-th identity feature and the m similarity thresholds, the m initial features are iteratively adjusted in multiple rounds to obtain m optimized features. Among these multiple rounds of iterative adjustments, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized. Different optimized features among the m optimized features represent different attribute information, and different optimized features among the m optimized features are used to generate different images of the i-th object.

2. The method according to claim 1, characterized in that, The step of obtaining the m similarity thresholds for the i-th object includes: The m similarity thresholds are sampled within a preset range of the similarity threshold values.

3. The method according to claim 1, characterized in that, The step of obtaining the m similarity thresholds for the i-th object includes: Select the m similarity thresholds that conform to a Gaussian distribution.

4. The method according to claim 1, characterized in that, The minimum difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds includes: When the m similarities include the similarity between each of the m optimized features and the i-th identity feature, the average of the m differences is minimized. The m differences include the differences between the similarity to the j-th optimized feature among the m similarities and the similarity threshold corresponding to the j-th optimized feature among the m similarity thresholds, where j is a positive integer less than or equal to m. The m similarities and the m optimized features have a one-to-one correspondence, and the m similarity thresholds also have a one-to-one correspondence with the m optimized features; or When the m similarities include the similarity between each of the m optimized features and the i-th identity feature, the mean square error of the m differences is minimized.

5. The method according to claim 1, characterized in that, Based on the i-th identity feature and the m similarity thresholds, the m initial features are iteratively adjusted in multiple rounds to obtain m optimized features, including: The m initial features are adjusted in the p-th iteration using the following steps, where p is a positive integer: Obtain the similarity between each of the m features adjusted in the (p-1)th round of iteration and the i-th identity feature, and obtain the m similarities in the p-th round. Where, when p equals 1, the m features adjusted in the (p-1)th round of iteration are the m initial features. Based on the m similarities in the p-th round and the m similarity thresholds, m differences in the p-th round are determined, wherein the m differences in the p-th round include the difference between the j-th similarity among the m similarities in the p-th round and the j-th similarity threshold among the m similarity thresholds, where j is a positive integer less than or equal to m. The m similarities in the p-th round have a one-to-one correspondence with the m features after iterative adjustment in the (p-1)-th round, and the m similarity thresholds have a one-to-one correspondence with the m features after iterative adjustment in the (p-1)-th round. Based on the m differences in the p-th round, determine the similarity loss value for the p-th round; If the similarity loss value does not meet the preset first termination adjustment condition, the m features after the (p-1)th iteration adjustment are adjusted to obtain the m features after the pth iteration adjustment. If the similarity loss value satisfies the first termination adjustment condition, the m features adjusted in the (p-1)th iteration are determined as the m optimized features.

6. The method according to any one of claims 1 to 5, characterized in that, The step of obtaining the identity feature of the i-th object among K objects, and obtaining the i-th identity feature, includes: Selecting the identity feature of the i-th object from K identity features yields the i-th identity feature, wherein the K identity features include the identity feature of each of the K objects, and each of the K identity features is used to characterize the identity information of the corresponding object among the K objects; or Select the identity feature of the i-th object from the K identity features, and normalize the identity feature of the i-th object to obtain the i-th identity feature.

7. The method according to claim 6, characterized in that, The method further includes: Obtain the initial identity features of each of the K objects to obtain K initial identity features; The K initial identity features are iteratively adjusted in multiple rounds to obtain the K identity features.

8. The method according to claim 7, characterized in that, The process of iteratively adjusting the K initial identity features to obtain the K identity features includes: The K initial identity features are adjusted through multiple rounds of iterations to obtain the K identity features, wherein the loss value determined based on the K identity features is the minimum during the multiple rounds of iterations.

9. The method according to claim 7, characterized in that, The process of iteratively adjusting the K initial identity features to obtain the K identity features includes: The K initial identity features are adjusted in the qth iteration using the following steps, where q is a positive integer: Based on the K features adjusted in the (q-1)th iteration, the loss value in the qth iteration is determined, wherein when q equals 1, the K features adjusted in the (q-1)th iteration include the K initial identity features; If the loss value in the qth round does not meet the preset second end adjustment condition, the K features after the iterative adjustment in the (q-1)th round are adjusted to obtain the K features after the iterative adjustment in the qth round. If the loss value in the qth round satisfies the second termination adjustment condition, the K features after the iterative adjustment in the (q-1)th round are determined as the K identity features.

10. The method according to claim 9, characterized in that, The process of determining the loss value for the qth round based on the K features adjusted in the (q-1)th iteration includes: Multiply the (q-1)th matrix with the transpose of the (q-1)th matrix to obtain the first matrix, wherein the (q-1)th matrix comprises a matrix composed of the K features adjusted in the (q-1)th iteration; Multiply the identity matrix by the preset coefficients to obtain the second matrix; Subtracting the second matrix from the first matrix yields the loss matrix for the qth round. The loss value of the qth round is determined based on the loss value matrix of the qth round.

11. The method according to claim 10, characterized in that, Determining the loss value of the qth round based on the loss value matrix of the qth round includes: Obtain the largest eigenvalue in each row of the loss matrix of the qth round, and obtain K eigenvalues; The loss value in the qth round is determined as the average or mean squared error of the K features.

12. The method according to claim 10, characterized in that, The adjustment of the K features after the (q-1)th iteration to obtain the K features after the qth iteration includes: Reduce at least some features among the K features adjusted in the (q-1)th iteration to obtain the K features adjusted in the qth iteration; or Decrease the third part of the K features after the (q-1)th iteration adjustment, and add the fourth part of the K features after the (q-1)th iteration adjustment to obtain the K features after the qth iteration adjustment, wherein the third part of the features is different from the fourth part of the features.

13. An image generation method, characterized in that, include: Obtain m optimized features and an initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature represents the identity information of the i-th object, where i is a positive integer. Each of the m initial features represents a type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multiple iterations, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized. Different optimized features represent different attribute information, and different optimized features are used to generate different images of the i-th object. Based on each of the m optimized features and the initial noisy image, m candidate images of the i-th object are generated.

14. The method according to claim 13, characterized in that, The step of generating m candidate images for the i-th object based on each of the m optimized features and the initial noisy image includes: One candidate image is generated from the m candidate images through the following steps: The candidate image is obtained by performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image.

15. The method according to claim 14, characterized in that, The process of performing multiple rounds of iterative processing on one of the m optimized features and the initial noisy image to obtain the candidate image includes: The following steps are used to perform the t-th iteration processing on one of the m optimized features and the initial noise image, where t is a positive integer: Based on the aforementioned optimization feature, the noise image after the (t-1)th iteration is denoised to obtain the first estimated noise image for the tth iteration, wherein when t=1, the noise image after the (t-1)th iteration is the initial noise image; Based on the aforementioned optimization feature, the noise image after the (t-1)th iteration is constrained to obtain the second estimated noise image for the tth iteration. Subtract the first estimated noise image and the second estimated noise image from the t-th round from the noise image after the (t-1)th iteration to obtain the noise image after the t-th iteration; If t reaches the preset number of iterations, the noise image after the t-th iteration is determined as a candidate image.

16. An image generation apparatus, characterized in that, include: The first acquisition unit is used to acquire the identity features of the i-th object among K objects to obtain the i-th identity feature, wherein the i-th identity feature is used to characterize the identity information of the i-th object, K is a positive integer greater than or equal to 2, and i is a positive integer less than or equal to K; The second acquisition unit is used to acquire m initial features of the i-th object, wherein each of the m initial features is used to characterize an object information of the i-th object, the object information including identity information and attribute information, and m is a positive integer greater than or equal to 2; The third acquisition unit is used to acquire m similarity thresholds for the i-th object; The first processing unit is configured to perform multiple rounds of iterative adjustments on the m initial features based on the i-th identity feature and the m similarity thresholds to obtain m optimized features. In the multiple rounds of iterative adjustments, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized. Different optimized features among the m optimized features represent different attribute information. Different optimized features among the m optimized features are used to generate different images of the i-th object.

17. An image generation apparatus, characterized in that, include: The fourth acquisition unit is used to acquire m optimized features and an initial noisy image of the i-th object. The m optimized features are obtained by iteratively adjusting the m initial features based on the i-th identity feature and m similarity thresholds. The i-th identity feature is used to characterize the identity information of the i-th object, where i is a positive integer. Each of the m initial features is used to characterize one type of object information of the i-th object, including identity information and attribute information, where m is a positive integer greater than or equal to 2. In the multiple rounds of iterative adjustment, the difference between the m similarities between the m optimized features and the i-th identity feature and the m similarity thresholds is minimized. Different optimized features in the m optimized features characterize different attribute information. Different optimized features in the m optimized features are used to generate different images of the i-th object. The second processing unit is used to generate m candidate images of the i-th object based on each of the m optimized features and the initial noise image.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored program, wherein the program can be executed by a terminal device or a computer to perform the method as described in any one of claims 1 to 12, or to perform the method as described in any one of claims 13 to 15.

19. A computer program product comprising a computer program / instructions, characterized in that, When executed by a processor, the computer program / instruction implements the steps of the method described in any one of claims 1 to 12, or performs the steps of the method described in any one of claims 13 to 15.

20. An electronic device comprising a memory and a processor, characterized in that, The memory stores a computer program, and the processor is configured to perform the method as described in any one of claims 1 to 12, or the method as described in any one of claims 13 to 15, through the computer program.

Images