Information processing system, information processing method, and program

Abstract

The present technology relates to an information processing system, an information processing method, and a program with which it is possible to easily obtain various variations of data. An acquisition unit acquires first data having a physical feature of a prescribed identity (ID). A generation unit imparts the first data as condition information to a generation model that generates data having a physical feature with noise as input data, thereby generating second data that has the physical feature of the prescribed ID and is different from the first data. The present technology can be applied, for example, when generating data such as face images in various variations.

Description

Information processing system, information processing method, and program

[0001] The present technology relates to an information processing system, an information processing method, and a program, and in particular to, for example, an information processing system, an information processing method, and a program that enable a wide variety of data to be easily obtained.

[0002] For example, training a learning model for face recognition requires a wide variety of facial images that represent (capture) the individual with each of the many IDs (identities) that identify (identify) an individual person (person). In other words, training a learning model requires a large number of facial images that represent (capture) the faces of various individuals in various poses, expressions, directions, lighting, and other conditions as training data.

[0003] Meanwhile, Article 6 of the General Data Protection Regulation (GDPR) [2] and Article 5 of the European AI Bill [3] state, "(db) the placing on the market, putting into service for this specific purpose, or use of AI systems that create or expand facial recognition databases through the untargeted scraping of facial images from the internet or CCTV footage;" This provision could potentially prohibit the use of images crawled (collected) from websites as training data for learning models.

[0004] This has led to problems such as the difficulty of building a licensed database of facial images with a wide variety of face images for each of a large number of IDs that can be used commercially in the future, and the high cost of building such a database.

[0005] Therefore, a technique has been proposed that generates various variations of images of the same individual by intentionally adding noise to the conditions given to the generative model (see, for example, Non-Patent Document 1).

[0006] Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, Romann M. Weber, "CADS: UNLEASHING THE DIVERSITY OF DIFFUSION MODELS THROUGH CONDITION-ANNEALED SAMPLING", Published as a conference paper at ICLR 2024

[0007] The technology described in Non-Patent Document 1 increases the diversity of variations in images depicting the same individual, while at the same time generating images in which individuality is not maintained, i.e., images that are not recognized (or cannot be determined) as belonging to the same ID (individual).

[0008] This technology was developed in light of these circumstances, and makes it possible to easily obtain data such as images of a wide variety of individuals with the same ID.

[0009] The information processing system or program of the present technology is an information processing system that includes an acquisition unit that acquires first data having physical characteristics of a predetermined ID (identity), and a generation unit that generates second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data, or a program for causing a computer to function as such an information processing system.

[0010] The information processing method of the present technology includes acquiring first data having physical characteristics of a predetermined ID (identity), and providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data, thereby generating second data that has the physical characteristics of the predetermined ID and is different from the first data.

[0011] In this technology, first data having physical characteristics of a predetermined ID (identity) is acquired, and the first data is provided as condition information to a generative model that generates data having the physical characteristics using noise as input data, thereby generating second data having the physical characteristics of the predetermined ID and different from the first data.

[0012] The information processing system may be an independent device or may be an internal block constituting a single device. Furthermore, one or more blocks constituting the information processing system may be configured as separate devices.

[0013] The program can be provided by transmitting it via a transmission medium or by recording it on a recording medium.

[0014] FIG. 1 is a block diagram showing an example configuration of an embodiment of a face recognition system to which the present technology is applied. FIG. 1 is a block diagram showing an example configuration of an information processing system as a face image generation unit 21. FIG. 2 is a diagram showing an example configuration of a generative model used in the face image generation unit 21. FIG. 3 is a diagram explaining an example of learning of a generative model 60. FIG. 4 is a diagram explaining an example of generation (inference) of a face image using a trained generative model 60. FIG. 5 is a flowchart explaining an example of processing by the face image generation unit 21. FIG. 6 is a diagram showing an example of input of a condition in a learning model. FIG. 7 is a diagram showing an example configuration of a generator 62 that constitutes the generative model 60. FIG. 8 is a diagram showing another example configuration of the generator 62 that constitutes the generative model 60. FIG. 9 is a diagram explaining an example of generation of data other than face images by the generative model 60. FIG. 10 is a block diagram showing an example configuration of a computer to which the present technology is applied.

[0015] <Facial recognition system applying this technology>

[0016] FIG. 1 is a block diagram showing an example of the configuration of an embodiment of a face recognition system to which the present technology is applied.

[0017] 1, a face recognition system 10 includes a learning device 20 and a recognition device 30. The learning device 20 performs learning for face recognition, i.e., learning of a learning model for face recognition. The recognition device 30 uses the learned learning model for face recognition to perform face recognition (inference / use of the learning model) of a face image in which a face is represented (shown).

[0018] The learning device 20 includes a face image generating unit 21 , a face image DB 22 , a learning face image selecting unit 23 , and a face image learning unit 24 .

[0019] The facial image generation unit 21 generates various facial images (data) and associates each facial image with an ID, as a label, that identifies the individual (person) of the person whose face is represented in the facial image. The facial image generation unit 21 supplies the facial images associated with the ID to the facial image DB 22 for storage (constructing the facial image DB 22 in which facial images associated with IDs are stored). In this embodiment, human individuals are used, but other animal individuals, such as dogs and cats, can also be used as individuals.

[0020] The face image DB 22 stores the face images from the face image generating unit 21 .

[0021] The training face image selection unit 23 selects face images (training face images) to be used as training data for a face recognition learning model from the face images stored in the face image DB 22, and supplies the selected face images to the face image learning unit 24. For example, the training face image selection unit 23 performs face recognition of the face images stored in the face image DB 22 using an existing trained classifier (learning model). As a result of the face recognition of the face images, the training face image selection unit 23 eliminates face images that are not recognized by the ID (individuals identified by the ID) associated with the face images (face images that do not maintain individuality (identity)), and selects the remaining face images as training data and supplies the selected face images to the face image learning unit 24.

[0022] If facial images that are not recognized by the individual with the ID associated with the facial image (the individual identified by the ID) are not eliminated, it becomes difficult to maintain the individuality of the facial images associated with the same ID (facial images that are not recognized by that ID (individual) will be mixed in with the facial images associated with the same ID). If facial images in which individuality is not maintained are used as training data to train a learning model, the accuracy of the learning model will decrease. On the other hand, if too many facial images are eliminated in the training facial image selection unit 23, the diversity (richness) of the variations in images depicting the same individual may be lost. There may be a trade-off between maintaining individuality and the diversity of the variations in images depicting the same individual.

[0023] The facial image learning unit 24 uses facial images as learning data from the learning facial image selection unit 23 to learn (including updating and fine-tuning) a learning model (classifier) ​​for face recognition, and supplies the learned learning model (trained learning model) to the recognition device 30.

[0024] The recognition device 30 includes a face detection unit 31 , a face image processing unit 32 , and a recognition result processing unit 33 .

[0025] A facial image of a target for facial recognition is supplied to the face detection unit 31. The face detection unit 31 detects a face depicted in the supplied facial image, identifies a predetermined area including the face, and supplies the area to the facial image processing unit 32.

[0026] The face image processing unit 32 is supplied with the face image in which the area including the face has been identified from the face detection unit 31, and is also supplied with the trained learning model supplied to the recognition device 30 from the face image learning unit 24. The face image processing unit 32 performs face recognition of the face image (the face depicted in the face image) from the face detection unit 31 using the learning model from the face image learning unit 24, and supplies the recognition result of the face recognition to the recognition result processing unit 33. In face recognition using the learning model, feature amounts are extracted from the face image, and the feature amounts are compared with the facial feature amounts of registered IDs to perform identification and matching processes based on conditions such as which ID the feature amounts are closest to or whether they are sufficiently close to those of the presented ID.

[0027] <Configuration example of face image generation unit 21>

[0028] FIG. 2 is a block diagram showing an example of the configuration of an information processing system serving as the face image generating unit 21 in FIG.

[0029] In FIG. 2, the face image generating unit 21 includes a learning device 40 and a generating device 50 .

[0030] The learning device 40 has a face image DB 41, an acquisition unit 42, and a learning unit 43, and performs learning of a generative model that generates face images. The learning device 40 supplies the learned generative model to the generation device 50.

[0031] The face image DB 41 stores face images used for training the generative model. Each face image is associated as a label with an ID that identifies the individual whose face is represented in the face image. For example, the face image DB 41 stores, for each of a plurality of IDs, a plurality of different face images associated with the ID. Different face images associated with the same ID are images that represent the face of an individual with the same ID, but have different facial poses (face orientations), facial expressions, lighting conditions, etc.

[0032] Here, a facial image representing the face of an individual with a specific ID is one type of data having the physical characteristics of the specific ID (physical characteristics identified by the specific ID). Other examples of data having the physical characteristics of the specific ID include an image representing the body of the individual with a specific ID, such as an image representing the whole body, and a 3D (dimensional) model of the individual with a specific ID, such as a mesh model.

[0033] The acquisition unit 42 acquires, from the face image DB 41, face images to be used for training a generative model in the training unit 43, and supplies the acquired face images to the training unit 43. For example, the acquisition unit 42 randomly selects two different face images with the same ID from the face image DB 41, and sets the combination of the two face images as training data for the generative model. The acquisition unit 42 supplies the training data to the training unit 43.

[0034] The learning unit 43 learns a learning model as a generative model using the learning data from the acquisition unit 42. For example, the learning unit 43 provides one of the combinations of face images to the generative model as condition information that becomes the condition of the generative model, and provides the other to the generative model as GT (Ground Truth) (correct answer) of the output data of the generative model, thereby learning the generative model.

[0035] As the generative model, a learning model that uses noise as input data and generates output data using conditions, such as a diffusion model or a learning model configured using GAN (Generative Adversarial Networks), can be adopted.

[0036] The learning unit 43 supplies the learned generative model, i.e., the generative model that has been trained using one of two different facial images representing an individual with the same ID as condition information and the other facial image as a GT, to the generating device 50.

[0037] The generating device 50 has a facial image DB 51, an acquisition unit 52, and a generating unit 53, and uses a generative model to generate facial images that are variations of the facial images stored in the facial image DB 51. That is, the generating device 50 generates different facial images (different facial images that represent (represent) individuals with the same ID) that represent (represent) the same individuals as those represented (represented) in the facial images stored in the facial image DB 51.

[0038] The face image DB 51 stores face images used to generate face images. Each face image is associated with an ID, which identifies an individual represented in the face image, as a label. For example, the face image DB 51 stores one or more face images associated with each of a plurality of IDs.

[0039] The acquisition unit 52 acquires facial images to be used for generating facial images from the facial image DB 51 and supplies them to the generation unit 53. For example, the acquisition unit 52 selects a predetermined ID, for example, an ID that has not yet been selected, as a target ID from the facial image DB 51 and selects and acquires one facial image of the target ID as a selected image. The acquisition unit 52 supplies the selected image (facial image of the target ID) to the generation unit 53.

[0040] The generation unit 53 generates a facial image (second data) different from the selected image by providing the selected image (first data) from the acquisition unit 52 as condition information to the generation model supplied from the learning device 40 to the generation device 50.

[0041] The generative model is a learning model that uses one of two different facial images representing individuals with the same ID as condition information and the other facial image as GT. Therefore, by providing the selected image as condition information to the generative model, the generative model generates a facial image that is different from the selected image and represents an individual with the same ID as the individual (whose face) represented in the selected image.

[0042] The generation unit 53 can randomly change the noise used as input data for the generative model. By changing the noise used as input data for the generative model to various noises, different facial images are generated in the generative model. That is, a facial image representing an individual with the same ID as the individual represented in the selected image, which is generated when the input data is a certain noise, is different from a facial image representing an individual with the same ID as the individual represented in the selected image, which is generated when the input data is a different noise.

[0043] In this way, a wide variety of data can be easily obtained with the generation device 50. That is, for example, by providing a selected image as condition information and changing the noise as input data to various noises, it is possible to obtain a wide variety of facial images that represent individuals with the same ID as the individual represented in the selected image.

[0044] Here, diffusion model-based methods, which are primarily used in generative AI (artificial intelligence), can generate desired data by providing features obtained from text prompts or reference images as conditions. In a diffusion model, by providing features from an individual's facial image as conditions, it is possible to generate a facial image that expresses (captures) that individual, but that differs from the facial image to which the features were provided as conditions. However, existing diffusion models tend to generate facial images with little intra-individual variation. "Low intra-individual variation" in facial images means that, within the range of different facial images that are determined to belong to individuals with the same ID, the degree of difference (the range of variation) is small, or the number of facial images that are determined to be substantially different is small.

[0045] By devising ways to assign conditions to the generative model during learning (training) of the generative model and during generation (inference) of facial images using the generative model, it is possible to easily generate a wide variety of facial images from a small group of facial images while maintaining individuality.

[0046] That is, the generative model is trained by selecting a combination of two facial images that represent the same individual from a group of facial images that differ in various variations (facial expressions, facial orientation, lighting, etc.) that represent each individual, and providing one facial image as condition information and the other facial image as the GT of the output data. In this way, the generative model is trained to generate a variety of images that represent the individual, regardless of the facial image that represents the individual that is provided as the condition. This makes it possible to easily generate a variety of facial images while maintaining individuality when generating facial images using the generative model.

[0047] For example, when facial images serving as training data for facial recognition training in the facial recognition system 10 of FIG. 1 are generated (augmented) using a generative model (AI), it may be difficult to obtain permission to use and collect a wide variety of facial images that represent individuals for each of a large number of IDs. For example, there may be a case where the number of IDs (individuals) is small. Alternatively, there may be a case where there are many IDs but only one facial image per ID, or where there are many facial images per ID but few substantially different facial images (many substantially identical facial images), resulting in little variation. Even in such cases, the facial image generation unit 21 can easily generate a wide variety of facial images while maintaining individuality.

[0048] <Example of generative model configuration>

[0049] FIG. 3 is a diagram showing an example of the configuration of a generation model used in the face image generation unit 21.

[0050] In FIG. 3, the generative model 60 includes a feature extractor 61 and a generator 62 .

[0051] Condition information that becomes a condition is supplied to the feature extractor 61. The feature extractor 61 extracts features from the condition information supplied thereto, such as latent variables in a latent space, and supplies them to the generator 62 as conditions.

[0052] For example, the feature extractor 61 can be a feature extractor that has been trained together with the generator 62. That is, (the parameters of) the feature extractor 61 can be trained together with the generator 62. In this case, the feature extractor 61 is trained to extract features specific to the facial image of each ID, and the facial image generated by the generative model 60 is an image that reflects the features specific to the facial image given as condition information. By fine-tuning the facial recognition system 10 using such facial images, the accuracy of identifying facial images can be improved.

[0053] Furthermore, for example, a trained VAE (Variational AutoEncoder) encoder can be used as the feature extractor 61. In this case, the feature amounts become latent variables output by the VAE encoder, and it is expected that among the latent variables (components of a vector), it is possible to extract (disentangle) ID-specific latent variables (components) and ID-independent latent variables.

[0054] Alternatively, for example, the feature extractor 61 may be a pre-trained model for face recognition using facial images, i.e., a feature extractor included in a learning model trained by the face recognition system 10 (the facial image learning unit 24 thereof). The feature extractor included in the learning model trained by the face recognition system 10 is trained to extract features appropriate for face recognition. Therefore, by employing such a feature extractor as the feature extractor 61, the features appropriate for face recognition can be used as conditions. Note that when a feature extractor included in a learning model trained by the face recognition system 10 is employed as the feature extractor 61, it is necessary to train the learning model for face recognition in advance using facial images acquired by any method.

[0055] The generator 62 uses noise as input data and the feature quantity from the feature extractor 61 as conditions, and generates a face image using the input data and the conditions. The generator 62 can be configured with a learning model, such as a diffusion model or GAN, that uses noise as input data and can generate data using the conditions.

[0056] <Learning a generative model>

[0057] FIG. 4 is a diagram illustrating an example of learning the generative model 60.

[0058] For example, face images with a small number of IDs (a small number of individuals) and a wide variety of face images (face images with diversity within an ID) that represent individuals with the same ID are stored in the face image DB 41. A small number of IDs means that there are a small number of individuals for whom usage permission must be obtained, so face images with a small number of IDs and a wide variety of face images that represent individuals with the same ID can be acquired relatively easily.

[0059] The generative model 60 is trained using the face image DB 41 (face images stored therein). That is, the acquisition unit 42 randomly selects two different face images with the same ID from the face image DB 41, and sets the combination of the two face images as training data for the generative model 60. The training unit 43 then trains the generative model 60 using the training data. For example, the training unit 43 provides one face image P1 of the combination of two different face images P1 and P2 with the same ID to the generative model 60 as condition information, and provides the other face image P2 to the generative model 60 as GT of output data, thereby training the generative model 60.

[0060] As a result, no matter what kind of facial image is given as condition information, the generative model 60 is trained to generate other facial images that represent the same individual (an individual determined to be the same) as the individual represented in the facial image. In other words, when a facial image is given as condition information, the generative model 60 is trained to generate a wide variety of facial images that represent the same individual as the individual represented in the facial image given as condition information by changing the noise as input data to various noises.

[0061] <Generating facial images using generative models>

[0062] FIG. 5 is a diagram illustrating an example of generation (inference) of a face image using a trained generative model 60.

[0063] The face image DB 51 stores face images with little variation (diversity) of face images representing individuals with the same ID, for example, a large number of IDs but only one face image per ID, etc. The face image DB 51 can also store face images stored in the face image DB 41.

[0064] The generation of a facial image using the generative model 60 is performed using the facial image DB 51 (facial images stored therein). That is, the acquisition unit 52 selects an arbitrary facial image from the facial image DB 51 as a selected image and supplies it to the generation unit 53. The generation unit 53 provides noise as input data to the generative model 60 and also provides the selected image as condition information. As a result, the generation unit 53 causes the generative model 60 to generate a facial image that is a variation of the selected image, i.e., a facial image that represents the same individual as the selected image (having the same ID as the selected image) but is different from the selected image.

[0065] The generative model 60 is trained using one of two different facial images representing individuals with the same ID as condition information and the other facial image as GT. By providing a selected image as condition information and changing the noise in the input data to various noises, the generative model 60 can easily generate a wide variety of facial images representing individuals with the same ID as the individual represented in the selected image.

[0066] In the above description, it is assumed that the facial images stored in the facial image DBs 41 and 51 are real-life facial images. However, the facial image DB 51 may store (data of) CG (computer graphics) images of rendered faces. In this case, the generative model 60 is trained using the real-life facial images stored in the facial image DB 41, and the facial images are generated using the CG images stored in the facial image DB 51 as selected images. When the facial images are generated using the CG images as selected images, the CG images are provided to the generative model 60 as condition information. This allows the generative model 60 to generate realistic facial images that are close to the real-life facial images used for training, while maintaining the individuality of the faces represented (rendered) in the CG images.

[0067] <Processing of the facial image generating unit 21>

[0068] FIG. 6 is a flowchart illustrating an example of processing by the face image generating unit 21 in FIG.

[0069] In step S11, the acquisition unit 42 randomly selects two different face images with the same ID from the face image DB 41, and sets many combinations of two different face images with the same ID as the learning data of the generative model 60. The acquisition unit 42 supplies the learning data to the learning unit 43, and the process proceeds from step S11 to step S12.

[0070] In step S12, the learning unit 43 provides one of the combinations of face images from the acquisition unit 42 to the generative model 60 as condition information that becomes the condition, and the other to the generative model 60 as GT of output data, thereby learning the generative model 60. After learning of the generative model 60 is completed, the learning unit 43 supplies the generative model 60 to the generation unit 53, and the process proceeds from step S12 to step S13.

[0071] In step S13, the acquisition unit 52 sequentially selects and acquires one face image for each ID as a selected image from the face image DB 51. The acquisition unit 52 supplies the selected image to the generation unit 53, and the process proceeds from step S13 to step S14.

[0072] The generation unit 53 provides the selected image from the acquisition unit 52 as condition information to the generation model 60 from the learning unit 43, and by changing the noise as input data of the generation model 60 to various noises, it generates, for each selected image, facial images of various variations of the facial image as the selected image (facial images with the same ID as the selected image but different from the selected image) as output data, and the processing then ends.

[0073] <Example of condition input>

[0074] FIG. 7 is a diagram showing an example of input of conditions in a learning model.

[0075] FIG. 7 shows an example of a network architecture for gControlNet (Generalized ControlNet) with a controllable normalization block (ControlNorm).

[0076] In Figure 7, conditions are generated from condition information in each encoder block in the upper layer, and the conditions are input to each controllable normalization block in the lower layer that configures a U-Net that uses noise as input data.

[0077] <Configuration example of generator 62>

[0078] FIG. 8 is a diagram showing an example of the configuration of the generator 62 that constitutes the generative model 60.

[0079] In FIG. 8, the generator 62 is composed of a diffusion model 71 .

[0080] The diffusion model 71 is a pixel-based diffusion model, and is configured, for example, by U-Net, etc. The pixel-based diffusion model 71 uses pixel-based noise as input data and conditions to generate a pixel-based denoised face image as output data.

[0081] In addition, the diffusion model 71 generates (estimates) noise superimposed on the facial image, rather than the facial image itself, and the generator 62 can generate a facial image by subtracting that noise from the input data.

[0082] FIG. 9 is a diagram showing another example of the configuration of the generator 62 that constitutes the generative model 60.

[0083] In FIG. 9, the generator 62 is composed of a latency diffusion model 81 and a decoder 82 .

[0084] The latent diffusion model 81 is a diffusion model that uses latent variables (or vectors) as input data and output data, and is configured, for example, by U-Net. The latent diffusion model 81 uses noise expressed in a space of latent variables (latent space), i.e., latent variables of noise, as input data, and generates a face image expressed in the latent space, i.e., a denoised latent variable of the face image, as output data using a condition. The latent diffusion model 81 supplies the denoised latent variable of the face image to the decoder 82.

[0085] The decoder 82 is configured, for example, by a VAE decoder, and generates a pixel-based face image from the latent variables from the latent diffusion model 81.

[0086] <Example of generating other data having physical characteristics>

[0087] FIG. 10 is a diagram illustrating an example of generation of data other than a facial image by the generative model 60.

[0088] In the above case, a facial image (selected image) of an individual with a specified ID as first data having physical characteristics is provided as condition information to generative model 60, which uses noise as input data, to generate a facial image as second data having the physical characteristics of the individual with the specified ID and different from the first data, i.e., a facial image that is a variation of the facial image as condition information. However, generative model 60 can generate data other than a facial image as second data.

[0089] In other words, the generative model 60 can generate, from first data having the physical characteristics of an individual with a specified ID, such as a facial image or a full-body image of the individual with a specified ID, second data having the physical characteristics of the individual with the specified ID that is different from the first data, i.e., data with a wide variety of variations that maintains the individuality of the individual with the specified ID.

[0090] For example, for face images of various IDs, a combination of a face image P and a 3D model M of the face represented in the face image P can be prepared as training data, and the training data can be used to train the generative model 60. As shown in Fig. 10 , the face image P from the combination of the face image P and the 3D model M can be provided to the generative model 60 as condition information, and the 3D model M can be provided to the generative model 60 as output data GT, thereby training the generative model 60.

[0091] As a result, the generative model 60 is trained to generate a 3D model of an individual represented in any facial image given as condition information.

[0092] As a result, in the generative model 60, when a facial image is given as condition information, various 3D models of the individual represented in the facial image given as condition information, for example, 3D models with different facial orientations and gaze directions, are generated by changing the noise as input data to various noises.

[0093] The generation model 60 for generating the above-described 3D model can be applied, for example, to generating a 3D image of a user's avatar in a game, or to generating a 3D image of a person looking in a desired direction.

[0094] In a combination of a facial image P and a 3D model M as learning data used to train the generative model 60, an appropriate texture can be associated with the 3D model M. The combination of the facial image P and the 3D model M associated with the texture can then be used as learning data to train the generative model 60. In this case, the trained generative model 60 can generate a 3D model of an individual represented in a facial image given as condition information, as well as a texture appropriate for the 3D model.

[0095] In addition, in FIG. 10, instead of the facial image showing the face, a whole-body image showing the whole body can be used.

[0096] FIG. 11 is a diagram illustrating another example of generation of data other than a facial image using the generative model 60. In FIG.

[0097] 11 , for full-body images representing individuals with various IDs, a combination of two different full-body images with the same ID is prepared as training data for the generative model 60, and training of the generative model 60 is performed. In Fig. 11 , one full-body image W1 of a combination of two different full-body images W1 and W2 with the same ID is provided to the generative model 60 as condition information, and the other full-body image W2 is provided to the generative model 60 as GT of output data, and training of the generative model 60 is performed.

[0098] As a result, the generative model 60 is trained to generate other whole-body images that depict the same individual (or an individual that is determined to be the same individual) as the individual depicted in the whole-body image, regardless of what kind of whole-body image is given as condition information.

[0099] As a result, in the generative model 60, when a whole-body image is given as condition information, by changing the noise as input data to various noises, a variety of whole-body images different from the whole-body image given as condition information are generated, which represent the same individual as the individual represented in the whole-body image given as condition information.

[0100] The generative model 60 that generates a wide variety of whole-body images as described above can be applied to, for example, generating training data used to train a learning model used in tasks such as person (human) re-identification.

[0101] <Description of a computer to which this technology is applied>

[0102] Next, the above-described series of processes can be performed by hardware or software. When the series of processes is performed by software, the programs that make up the software are installed on a general-purpose computer or the like.

[0103] FIG. 12 is a block diagram showing an example of the configuration of an embodiment of a computer in which a program for executing the above-described series of processes is installed.

[0104] The program can be recorded in advance on the hard disk 905 or ROM 903 as a recording medium built into the computer.

[0105] Alternatively, the program can be stored (recorded) on a removable recording medium 911 driven by the drive 909. Such a removable recording medium 911 can be provided as a so-called package software. Here, examples of the removable recording medium 911 include a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

[0106] The program can be installed into the computer from the removable recording medium 911 as described above, or can be downloaded to the computer via a communication network or a broadcasting network and installed on the built-in hard disk 905. That is, the program can be transferred to the computer wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transferred to the computer via a wired network such as a LAN (Local Area Network) or the Internet.

[0107] The computer includes a CPU (Central Processing Unit) 902 , to which an input / output interface 910 is connected via a bus 901 .

[0108] When a user inputs a command via an input / output interface 910 by operating an input unit 907, the CPU 902 executes a program stored in a read-only memory (ROM) 903 in accordance with the command. Alternatively, the CPU 902 loads a program stored on a hard disk 905 into a random access memory (RAM) 904 and executes the program.

[0109] As a result, the CPU 902 performs processing according to the flowchart described above or processing performed by the configuration of the block diagram described above. Then, the CPU 902 outputs the processing results from the output unit 906 via the input / output interface 910, or transmits them from the communication unit 908, or further records them on the hard disk 905, as necessary.

[0110] The input unit 907 is made up of a keyboard, a mouse, a microphone, etc. The output unit 906 is made up of an LCD (Liquid Crystal Display), a speaker, etc.

[0111] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0112] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0113] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0114] It should be noted that the embodiments of the present technology are not limited to the above-described embodiments, and various modifications are possible within the scope of the present technology.

[0115] For example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0116] Furthermore, each step described in the above flowchart can be executed by one device, or can be shared and executed by a plurality of devices.

[0117] Furthermore, when one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices.

[0118] Furthermore, the effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0119] The present technology can have the following configurations.

[0120] <1> An information processing system comprising: an acquisition unit that acquires first data having physical characteristics of a predetermined ID (identity); and a generation unit that generates second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data. <2> The generation unit generates a plurality of the second data by changing the noise. <3> The information processing system described in <1> or <2>, wherein the first data is image data representing an individual with the predetermined ID, and the second data is image data representing the individual with the predetermined ID but different from the first data. <4> The information processing system described in <3>, wherein the first data is facial image data representing a face of the individual with the predetermined ID, and the second data is facial image data representing the face of the individual with the predetermined ID but different from the first data. <5> The information processing system described in <3> or <4>, wherein the generative model is a learning model that uses one image data of a combination of two different image data representing individuals with the same ID as the condition information and the other image data as GT (Ground Truth). <6> The information processing system described in <5>, wherein the combination of the two image data is generated using a DB (database) that stores image data with a wide variety of variations representing individuals with the same ID. <7> The information processing system described in <4>, wherein the generative model includes: a generator that generates facial image data using the noise and the condition; and a feature extractor that extracts features from the condition information and provides the extracted features to the generator as the condition. <8> The information processing system described in <7>, wherein the feature extractor is a feature extractor that has been trained together with the generator. <9> The information processing system described in <7>, wherein the feature extractor is an encoder for a VAE (Variational AutoEncoder).<10> The information processing system according to <7>, wherein the feature extractor is the same feature extractor as that included in a trained learning model for face recognition using facial images. <11> The information processing system according to any one of <7> to <10>, wherein the generator is configured as a diffusion model. <12> The information processing system according to <11>, wherein the generative model is configured as a pixel-based diffusion model. <13> The information processing system according to <11>, wherein the generative model is configured as a latent diffusion model. <14> The information processing system according to <1> or <2>, wherein the first data is image data representing an individual having the specified ID, and the second data is a 3D (dimensional) model of the individual having the specified ID. <15> The information processing system according to any one of <1> to <14>, wherein the first data is image data of a real photograph. <16> The information processing system according to any one of <1> to <14>, wherein the first data is CG (computer graphics) image data. <17> An information processing method including: acquiring first data having physical characteristics of a predetermined ID (identity); and generating second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data. <18> A program for causing a computer to function as an information processing system including: an acquisition unit that acquires first data having physical characteristics of a predetermined ID (identity); and a generation unit that generates second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data.

[0121] 10 Face recognition system, 20 Learning device, 21 Face image generation unit, 22 Face image DB, 23 Learning face image selection unit, 24 Face image learning unit, 30 Recognition device. 31 Face detection unit, 32 Face image processing unit, 33 Recognition result processing unit, 40 Learning device, 41 Face image DB, 42 Acquisition unit, 43 Learning unit, 50 Generation device, 51 Face image DB, 52 Acquisition unit, 53 Generation unit, 60 Generative model, 61 Feature extractor, 62 Generator, 71 Diffusion model, 81 Latent diffusion model, 82 Decoder, 901 Bus, 902 CPU, 903 ROM, 904 RAM, 905 Hard disk, 906 Output unit, 907 Input unit, 908 Communication unit, 909 Drive, 910 Input / output interface, 911 Removable recording medium

Claims

1. An information processing system comprising: an acquisition unit that acquires first data having physical characteristics of a predetermined ID (identity); and a generation unit that generates second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data.

2. The information processing system according to claim 1, wherein the generating unit generates a plurality of the second data by changing the noise.

3. The information processing system according to claim 1, wherein the first data is image data representing an individual with the specified ID, and the second data is image data different from the first data representing an individual with the specified ID.

4. The information processing system according to claim 3, wherein the first data is facial image data representing the face of the individual with the specified ID, and the second data is facial image data different from the first data representing the face of the individual with the specified ID.

5. The information processing system according to claim 3, wherein the generative model is a learning model that uses one image data of a combination of two different image data representing individuals with the same ID as the condition information and the other image data as GT (Ground Truth).

6. The information processing system according to claim 5, wherein the combination of the two image data is selected from a database that stores a wide variety of image data that represent individuals with the same ID.

7. The information processing system according to claim 4, wherein the generative model includes: a generator that generates facial image data using the noise and the condition; and a feature extractor that extracts features from the condition information and provides them to the generator as the condition.

8. The information processing system according to claim 7, wherein the feature extractor is a feature extractor trained together with the generator.

9. The information processing system according to claim 7, wherein the feature extractor is a VAE (Variational AutoEncoder) encoder.

10. The information processing system according to claim 7, wherein the feature extractor is the same feature extractor as that included in a trained learning model for face recognition using face images.

11. The information processing system according to claim 7, wherein the generator is configured using a diffusion model.

12. The information processing system according to claim 11, wherein the generative model is a pixel-based diffusion model.

13. The information processing system according to claim 11, wherein the generative model is configured as a latent diffusion model.

14. The information processing system according to claim 1, wherein the first data is image data representing an individual with the specified ID, and the second data is a 3D (dimensional) model of the individual with the specified ID.

15. The information processing system according to claim 1, wherein the first data is image data of a real photograph.

16. The information processing system according to claim 1, wherein the first data is CG (computer graphics) image data.

17. An information processing method comprising: acquiring first data having physical characteristics of a predetermined ID (identity); and generating second data having the physical characteristics of the predetermined ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data.

18. A program for causing a computer to function as an information processing system comprising: an acquisition unit that acquires first data having the physical characteristics of a specified ID (identity); and a generation unit that generates second data having the physical characteristics of the specified ID and different from the first data by providing the first data as condition information to a generative model that generates data having the physical characteristics using noise as input data.

Images