Image generation method and apparatus, electronic device, and storage medium

Abstract

The application relates to an image generation method and device, electronic equipment and a storage medium, and applies to the technical field of data processing. The method comprises the following steps: acquiring an initial feature vector of a target object; acquiring a classification boundary used for judging the category of an additional feature, wherein the additional feature is a feature irrelevant to the facial organ feature of the target object; adjusting the initial feature vector according to the classification boundary to obtain each target feature vector of the additional feature belonging to different categories; and generating an image of the target object corresponding to each target feature vector. The application solves the problem that in the prior art, the training data set of multiple morphological attributes of the same face is often obtained from multiple parties, and the acquisition process is complex and difficult.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an image generation method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of artificial intelligence, facial recognition based on neural network models is being used more and more widely.

[0003] In the process of facial recognition, the training dataset often needs to include a large amount of image data of the same face from multiple angles, with different clothing and styles, in order to achieve accurate facial recognition.

[0004] In related technologies, obtaining a training dataset with multiple morphological attributes of the same face often requires acquisition from multiple sources, a process that is complex and difficult. Summary of the Invention

[0005] This application provides an image generation method, apparatus, electronic device, and storage medium to solve the problem in the prior art that obtaining training datasets with multiple morphological attributes of the same face often requires acquisition from multiple sources, and the acquisition process is complex and difficult.

[0006] In a first aspect, embodiments of this application provide an image generation method, including:

[0007] Obtain the initial feature vector of the target object;

[0008] Obtain the classification boundary used to determine the category to which the additional feature belongs, wherein the additional feature is a feature unrelated to the facial organ features of the target object;

[0009] Based on the classification boundary, the initial feature vector is adjusted to obtain target feature vectors for each category to which the additional features belong;

[0010] Generate an image of the target object corresponding to each of the target feature vectors.

[0011] Optionally, adjusting the initial feature vector according to the classification boundary to obtain target feature vectors for which the additional features belong to different categories includes:

[0012] Obtain the relative parameters between the initial feature vector and the classification boundary, the relative parameters including distance and / or direction;

[0013] Adjust the relative parameters to obtain the target relative parameters;

[0014] The target feature vector is determined based on the target relative parameters.

[0015] Optionally, adjusting the relative parameters to obtain the target relative parameters includes:

[0016] Obtain the adjustment value of the relative parameter, wherein the adjustment value falls within a preset adjustment range;

[0017] The target relative parameter is obtained after adjusting the relative parameter according to the adjustment value.

[0018] Optionally, obtaining the classification boundary for determining the category to which the additional features belong, wherein the additional features are features unrelated to the facial organ features of the target object, includes:

[0019] A classification model is trained using a sample set with additional features, wherein the sample set includes sample images with the additional features and the identifiers of the additional features;

[0020] From the classification model, obtain the classification boundary used to determine the category to which the additional feature belongs.

[0021] Optionally, the classification model is trained using a sample set with additional features, wherein the sample set includes sample images with the additional features and identifiers of the additional features, including:

[0022] Obtain a sample set, which includes sample images and identifiers of additional features in the sample images;

[0023] Perform the following training process on each sample image in the sample image set:

[0024] The sample image is input into an initial classification model, which includes a classification kernel function. The initial classification model is used to extract the sample feature vector of the sample image, and the sample feature vector is classified to obtain the classification result.

[0025] The loss function value is calculated based on the classification result and the additional feature identifier;

[0026] Determine whether the loss function value is less than a preset value. If yes, use the initial classification model as the final classification model and the classification kernel function as the target classification kernel function. If no, adjust the parameters of the initial classification model according to the loss function value and repeat the training process until the loss function value is less than the preset value.

[0027] Optionally, obtaining the classification boundary from the classification model includes:

[0028] Obtain the target classification kernel function in the classification model;

[0029] The target classification kernel function whose value is the target boundary value is used as the classification boundary.

[0030] Optionally, the sample image is an initial image of at least one of the target objects, the initial image being generated based on the initial feature vector.

[0031] Optionally, obtaining the initial feature vector of the target object includes:

[0032] Obtain an initial image of the target object, transform the initial image to obtain the initial feature vector; or,

[0033] The initial feature vector is obtained from a pre-trained object generation model.

[0034] Optionally, generating the image of the target object corresponding to each of the target feature vectors includes:

[0035] The target feature vector is input into a pre-trained object generation model.

[0036] An image of the target object is generated using the object generation model.

[0037] Secondly, embodiments of this application provide an image generation apparatus, comprising:

[0038] The first acquisition module is used to acquire the initial feature vector of the target object;

[0039] The second acquisition module is used to acquire the classification boundary for determining the category to which the additional feature belongs, wherein the additional feature is a feature that is unrelated to the facial organ features of the target object;

[0040] An adjustment module is used to adjust the initial feature vector according to the classification boundary to obtain target feature vectors of different categories to which the additional features belong;

[0041] The generation module is used to generate an image of the target object corresponding to each of the target feature vectors.

[0042] Thirdly, embodiments of this application provide an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0043] The memory is used to store computer programs;

[0044] The processor is configured to execute the program stored in the memory to implement the image generation method described in the first aspect.

[0045] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the image generation method described in the first aspect.

[0046] Compared with the prior art, the above-mentioned technical solution provided in this application has the following advantages: The method provided in this application obtains the initial feature vector of the target object and the classification boundary for determining the category to which the additional features belong. Then, according to the classification boundary, the initial feature vector is adjusted to obtain target feature vectors of different categories to which the additional features belong. Since the additional features are features unrelated to the facial organ features of the target object, the generated target feature vectors will not change the facial organ features of the target object. Moreover, it is only necessary to obtain the initial feature vector of the target object and process it accordingly to determine the target feature vector, thereby generating the image of the target object corresponding to each target feature vector. In this way, the acquisition path of the image of the same target object is a single path acquisition, and the acquisition process is more convenient. Attached Figure Description

[0047] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0048] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0049] Figure 1 A flowchart illustrating an image generation method provided in an embodiment of this application;

[0050] Figure 2 A flowchart of an image generation method provided in another embodiment of this application;

[0051] Figure 3 This is a structural diagram of an image generation apparatus provided in an embodiment of this application;

[0052] Figure 4 This is a structural diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0054] One embodiment of this application provides an image generation method, which can be applied to any type of electronic device, such as a terminal and a server. Figure 1 As shown, the image generation method includes:

[0055] Step 101: Obtain the initial feature vector of the target object.

[0056] In some embodiments, the target object may be a person or an animal, and this application does not limit it.

[0057] Furthermore, there are multiple ways to obtain the initial feature vector, which will be illustrated by the following two examples.

[0058] The first method involves obtaining the initial image of the target object, transforming the initial image, and obtaining the initial feature vector.

[0059] The initial image of the target object can be captured by a camera or generated by an object generation model. If the initial image is captured by a camera, there are several ways to convert the initial image into an initial feature vector. For example, the Histogram of Oriented Gradients (HOG) feature extraction method can be used.

[0060] The second method involves obtaining initial feature vectors from a pre-trained object generation model.

[0061] When the initial image is generated from the object generation model, the initial feature vector corresponding to the initial image can be obtained from the input of the object generation model.

[0062] It is understandable that when the target object is a human, the initial image can be a human face image, and the object generation model can be a human face generation model; when the target object is an animal, the initial image can be an animal head image, and the object generation model can be a head generation model.

[0063] Step 102: Obtain the classification boundary used to determine the category to which the additional features belong. The additional features are features that are unrelated to the facial organ features of the target object.

[0064] In some embodiments, when the target object is a person, the additional features include, but are not limited to, at least one of the following: human hair features, glasses features, scarf features, and clothing features; when the target object is an animal, the additional features include, but are not limited to, at least one of the following: animal hair length and clothing features.

[0065] There are several ways to obtain the classification boundary used to determine the category to which the additional features belong. For example, a classification model can be trained using a sample set of additional features, which includes sample images with additional features and the labels of the additional features; the classification boundary can then be obtained from the classification model.

[0066] The classification model can be trained based on various initial models, such as binary classification models like support vector machines (SVM).

[0067] Specifically, the classification model can be trained in the following ways:

[0068] Step 201: Obtain a sample set, which includes sample images and additional feature identifiers of the sample images. The additional feature identifiers are used to indicate additional features in the sample images.

[0069] Perform the following training process for each sample image in the sample image set:

[0070] Step 202: Input the sample image into the initial classification model, which includes a classification kernel function. Extract the sample feature vector of the sample image through the initial classification model, and classify the sample feature vector based on the classification kernel function to obtain the classification result.

[0071] Step 203: Calculate the loss function value based on the classification results and additional feature identifiers.

[0072] Step 204: Determine whether the loss function value is less than the preset value; if not, proceed to step 205; if yes, proceed to step 206.

[0073] Step 205: Adjust the parameters of the initial classification model based on the loss function value, and return to execute step 202.

[0074] Step 206: Use the initial classification model as the classification model and the classification kernel function as the target classification kernel function.

[0075] Furthermore, the target classification kernel function whose value is the target boundary value can be used as the classification boundary. It's understandable that the target boundary value can be obtained from the training results of the classification model. When the classification kernel function is equal to the target boundary value, it represents the classification boundary; when the classification kernel function is greater than or less than the target boundary value, they represent two categories with the same dimension of the additional features. For example, if the target boundary value is 0, then a classification kernel function of 0 represents the classification boundary; a classification kernel function greater than 0 indicates the target object has long hair, and a classification kernel function less than 0 indicates the target object has short hair.

[0076] The classification kernel function is a non-linear kernel function, and the sample image is an initial image of at least one target object, which is generated based on the initial feature vector.

[0077] In this embodiment, to make the classification boundary determined by the classification model more accurate, sample images can be generated by an object generation model. That is, a large number of initial images are generated by the object generation model, and then the classification model is used to classify the images to obtain a more accurate classification boundary.

[0078] Step 103: Adjust the initial feature vector according to the classification boundary to obtain target feature vectors whose additional features belong to different categories.

[0079] In some embodiments, after determining the classification boundary, since the classification boundary can be used to determine the category to which the additional features belong, adjusting the initial feature vector according to the classification boundary can make the additional features of the obtained target feature vector different from the initial feature vector.

[0080] Specifically, based on the classification boundary, the initial feature vector is adjusted to obtain additional feature vectors for each target category, including:

[0081] Obtain the relative parameters between the initial feature vector and the classification boundary, including distance and / or direction; adjust the relative parameters to obtain the target relative parameters; determine the target feature vector based on the target relative parameters.

[0082] In some embodiments, to determine the target feature vector, the relative parameters between the initial feature vector and the classification boundary can be obtained first. When adjusting the relative parameters, at least one of distance and direction can be adjusted to obtain the target relative parameters. Then, the values ​​in the initial feature vector can be adjusted through the target relative parameters to obtain the target feature vector.

[0083] The relative parameters can be obtained by calculating the distance and direction between the initial feature vector and the normal vector of the classification boundary.

[0084] Furthermore, by adjusting the relative parameters, the target relative parameters are obtained, including:

[0085] Obtain the adjustment value of the relative parameter, which falls within the preset adjustment range; obtain the target relative parameter after adjusting the relative parameter according to the adjustment value.

[0086] In some embodiments, the adjustment value can be at least one of distance and direction. When adjusting the relative parameters, the adjustment value is adjusted within a preset adjustment range to avoid excessive adjustment that could lead to image distortion. Image distortion refers to the generated image not corresponding to the same target object as the initial image generated from the initial feature vector.

[0087] Step 104: Generate the image of the target object corresponding to each target feature vector.

[0088] The generated image includes the target object and additional features.

[0089] In some embodiments, there are multiple ways to generate images from target feature vectors. For example, the target feature vector can be input into a pre-trained object generation model, and the object generation model can generate an image of the target object.

[0090] Specifically, there are several ways to generate an image of a target object from its feature vector. The target feature vector can be input into an object generation model, which then generates the image of the target object. Taking a person as an example, the object generation model could be the StyleGAN (Generative Adversarial Network) model. The input to the StyleGAN model is a 1*512 dimensional feature vector.

[0091] StyleGAN is a neural network that takes feature vectors as input and face images as output. It can control high-level attributes of the generated images, such as hairstyle and freckles. The generated images score better on some evaluation criteria and can generate different face images based on different input feature vectors.

[0092] In one specific embodiment, taking a person as the target object and a face image as the example, a detailed explanation will be provided.

[0093] In this embodiment, the user can set N initial feature vectors and input each initial feature vector into a pre-trained StyleGAN model to generate N initial face images. The people in each initial face image are different, and N is a positive integer greater than 1.

[0094] After obtaining each initial face image, additional features are acquired in different dimensions of each initial face image of that person. For example, whether the person in the initial face image wears glasses or not, whether the hair is long or short, and whether the hair color is black or other colors. The initial face images labeled with additional features are input into the SVM, and the classification kernel function of the SVM is set to complete the training of the SVM, and the classification boundary of each feature dimension is obtained.

[0095] In the trained SVM, the classification boundary is defined when the classification kernel function is 0. The distance or direction of each initial feature vector to the classification boundary is adjusted to obtain the target feature vector. This target feature vector is then input into the StyleGAN model to obtain other face images of the individuals in the initial face images.

[0096] Understandably, in order to obtain more facial images of the target person, the distance or direction between the initial feature vector and different classification boundaries can be adjusted multiple times to obtain a large number of facial images of the target person. Thus, it is not necessary to obtain them from multiple sources or to take multiple photos of the same person from different angles, which can provide a rich dataset for facial recognition.

[0097] In this application, an initial feature vector of the target object is obtained, along with a classification boundary used to determine the category of additional features. Based on this boundary, the initial feature vector is adjusted to obtain target feature vectors for each category. Since the additional features are unrelated to the facial features of the target object, the generated target feature vectors do not alter these features. Furthermore, only the initial feature vector of the target object needs to be obtained and processed accordingly to determine the target feature vectors, thereby generating images of the target object corresponding to each feature vector. This makes the acquisition of images of the same target object a single-path acquisition, making the process more convenient. This application allows for the acquisition of multi-dimensional face images, providing a richer dataset for face recognition.

[0098] Based on the same concept, this application provides an image generation apparatus in its embodiments. Specific implementation of this apparatus can be found in the description of the method embodiments section; repeated details will not be repeated here. Figure 3 As shown, the device mainly includes:

[0099] The first acquisition module 301 is used to acquire the initial feature vector of the target object;

[0100] The second acquisition module 302 is used to acquire the classification boundary for determining the category to which the additional features belong, wherein the additional features are features that are unrelated to the facial organ features of the target object;

[0101] The adjustment module 303 is used to adjust the initial feature vector according to the classification boundary to obtain target feature vectors whose additional features belong to different categories;

[0102] The generation module 304 is used to generate the image of the target object corresponding to each target feature vector.

[0103] In this embodiment, the initial feature vector of the target object is obtained by the first acquisition module, and the classification boundary used to determine the category to which the additional features belong is obtained by the second acquisition module. Then, the adjustment module adjusts the initial feature vector according to the classification boundary to obtain target feature vectors of different categories to which the additional features belong. Since the additional features are features unrelated to the facial organ features of the target object, the generated target feature vectors will not change the facial organ features of the target object. Furthermore, only the initial feature vector of the target object needs to be obtained and processed accordingly to determine the target feature vectors. Thus, the generation module generates the image of the target object corresponding to each target feature vector. In this way, the acquisition of the image of the same target object is achieved through a single path, making the acquisition process more convenient. This application can obtain multi-dimensional face images, providing a richer dataset for face recognition.

[0104] Based on the same concept, this application also provides an electronic device, such as... Figure 4 As shown, the electronic device mainly includes a processor 401, a memory 402, and a communication bus 403. The processor 401 and the memory 402 communicate with each other via the communication bus 403. The memory 402 stores programs that can be executed by the processor 401. The processor 401 executes the programs stored in the memory 402 to perform the following steps:

[0105] Obtain the initial feature vector of the target object;

[0106] Obtain the classification boundary used to determine the category to which the additional feature belongs, wherein the additional feature is a feature unrelated to the facial organ features of the target object;

[0107] Based on the classification boundary, the initial feature vector is adjusted to obtain target feature vectors for each category to which the additional features belong;

[0108] Generate an image of the target object corresponding to each of the target feature vectors.

[0109] The communication bus 403 mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus 403 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0110] The memory 402 may include random access memory (RAM) or non-volatile memory, such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor 401.

[0111] The processor 401 mentioned above can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc., or a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0112] In another embodiment of this application, a computer-readable storage medium is provided, which stores a computer program that, when run on a computer, causes the computer to perform the image generation method described in the above embodiments.

[0113] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. This computer program product includes one or more computer instructions. When these computer instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another, for example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape, etc.), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive), etc.

[0114] It should be noted that, in this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0115] The above description is merely a specific embodiment of the present invention, enabling those skilled in the art to understand or implement the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the present invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features claimed herein.

Claims

1. An image generation method, characterized in that, include: Obtain the initial feature vector of the target object; Obtain the classification boundary used to determine the category to which the additional feature belongs, wherein the additional feature is a feature unrelated to the facial organ features of the target object; Based on the classification boundary, the initial feature vector is adjusted to obtain target feature vectors for each category to which the additional features belong; Generate an image of the target object corresponding to each of the target feature vectors; The step of obtaining the classification boundary for determining the category to which the additional feature belongs, wherein the additional feature is a feature unrelated to the facial organ features of the target object, includes: training a classification model using a sample set of additional features, wherein the sample set includes sample images with the additional features and the identifiers of the additional features; and obtaining the classification boundary for determining the category to which the additional feature belongs from the classification model.

2. The image generation method according to claim 1, characterized in that, The step of adjusting the initial feature vector according to the classification boundary to obtain target feature vectors for each category to which the additional features belong includes: Obtain the relative parameters between the initial feature vector and the classification boundary, the relative parameters including distance and / or direction; Adjust the relative parameters to obtain the target relative parameters; The target feature vector is determined based on the target relative parameters.

3. The image generation method according to claim 2, characterized in that, The adjustment of the relative parameters to obtain the target relative parameters includes: Obtain the adjustment value of the relative parameter, wherein the adjustment value falls within a preset adjustment range; The target relative parameter is obtained after adjusting the relative parameter according to the adjustment value.

4. The image generation method according to claim 3, characterized in that, The classification model is trained using a sample set with additional features. The sample set includes sample images with the additional features and identifiers for the additional features, including: Obtain a sample set, which includes sample images and identifiers of additional features in the sample images; Perform the following training process on each sample image in the sample image set: The sample image is input into an initial classification model, which includes a classification kernel function. The initial classification model is used to extract the sample feature vector of the sample image, and the sample feature vector is classified to obtain the classification result. The loss function value is calculated based on the classification result and the additional feature identifier; Determine whether the loss function value is less than a preset value. If yes, use the initial classification model as the final classification model and the classification kernel function as the target classification kernel function. If no, adjust the parameters of the initial classification model according to the loss function value and repeat the training process until the loss function value is less than the preset value.

5. The image generation method according to claim 4, characterized in that, Obtaining the classification boundary from the classification model includes: Obtain the target classification kernel function in the classification model; The target classification kernel function whose value is the target boundary value is used as the classification boundary.

6. The image generation method according to claim 3, characterized in that, The sample image is an initial image of at least one of the target objects, and the initial image is generated based on the initial feature vector.

7. The image generation method according to claim 1, characterized in that, The process of obtaining the initial feature vector of the target object includes: Obtain an initial image of the target object, transform the initial image to obtain the initial feature vector; or, The initial feature vector is obtained from a pre-trained object generation model.

8. The image generation method according to claim 1, characterized in that, Generating the image of the target object corresponding to each of the target feature vectors includes: The target feature vector is input into a pre-trained object generation model. An image of the target object is generated using the object generation model.

9. An image generation apparatus, characterized in that, include: The first acquisition module is used to acquire the initial feature vector of the target object; The second acquisition module is used to acquire the classification boundary for determining the category to which the additional feature belongs, wherein the additional feature is a feature that is unrelated to the facial organ features of the target object; An adjustment module is used to adjust the initial feature vector according to the classification boundary to obtain target feature vectors of different categories to which the additional features belong; The generation module is used to generate an image of the target object corresponding to each of the target feature vectors; The step of obtaining the classification boundary for determining the category to which the additional feature belongs, wherein the additional feature is a feature unrelated to the facial organ features of the target object, includes: training a classification model using a sample set of additional features, wherein the sample set includes sample images with the additional features and the identifiers of the additional features; and obtaining the classification boundary for determining the category to which the additional feature belongs from the classification model.

10. An electronic device, characterized in that, include: The processor, communication interface, memory, and communication bus are connected, with the processor, communication interface, and memory communicating with each other via the communication bus. The memory is used to store computer programs; The processor is configured to execute the program stored in the memory to implement the image generation method according to any one of claims 1-8.

11. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the image generation method according to any one of claims 1-8.

Images