DATA GENERATION METHOD, DATA GENERATION DEVICE, AND PROGRAM
The image generation device fuses latent information of two images using genetic and logical operations within a generative model to create unique and harmonious images, addressing the limitations of existing systems by enhancing variation and coherence.
Patent Information
- Application Number
- JP2023099732
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2019-03-11
- Filing Date
- 2023-06-19
- Publication Date
- 2024-11-13
- Estimated Expiration
- 2039-03-18
AI Technical Summary
Existing image generation systems, such as CryptoKitties, produce images with limited unique features and little variation by simply combining parts of parent images, lacking creativity and harmony.
An image generation device that fuses latent information, including genetic codes and attributes, of two images using a generative model, employing operations like crossover, mutation, and logical/arithmetic operations to create a unique and harmonious fused image.
Generates images with rich variation and harmony by inheriting characteristics from both input images, using a trained generative model to produce a cohesive output.
Smart Images

Figure 0007582592000001 
Figure 0007582592000002 
Figure 0007582592000003
Abstract
Description
[Technical field]
[0001] The present disclosure relates to an image generation method. [Background technology]
[0002] On networks such as the Internet, sites or platforms that provide various tools are provided. For example, a platform has been established on the Internet that provides tools that allow users to create digital images such as avatar images and character images using provided parts, and to edit, publish, exchange, and the like, the created digital images.
[0003] For example, in CryptoKitties, users can create an image of a baby cat from images of a parent cat, and can buy and sell the images of cats they own using virtual currency.
[0004] However, the images of baby cats generated in CryptoKitties were generated by combining each part (e.g., eyes, ears, etc.) of the parent and parent cats, and therefore lacked unique features and little variation. [Prior art documents] [Non-patent literature]
[0005] [Non-Patent Document 1] https: / / www.cryptokitties.co / Summary of the Invention [Problem to be solved by the invention]
[0006] An object of the present disclosure is to provide a novel image generation technique. [Means for solving the problem]
[0007] In order to solve the above problem, one aspect of the present disclosure is a method for detecting a plurality of latent information. of Generative Model By entering The present invention relates to a data generation method, comprising: generating data, the latent information being information represented by a blockchain-compliant code. [Brief description of the drawings]
[0008] [Figure 1] FIG. 2 is a schematic diagram illustrating an image generation process according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 illustrates a digital image according to one embodiment of the present disclosure. [Diagram 3] FIG. 2 illustrates a digital image according to one embodiment of the present disclosure. [Figure 4] FIG. 2 illustrates a digital image according to one embodiment of the present disclosure. [Diagram 5] FIG. 1 is a schematic diagram illustrating an image generating device according to an embodiment of the present disclosure. [Figure 6] FIG. 2 is a block diagram showing a functional configuration of an image generating device according to an embodiment of the present disclosure. [Figure 7] FIG. 1 illustrates image information on a platform according to one embodiment of the present disclosure. [Figure 8] FIG. 1 illustrates image information on a platform according to one embodiment of the present disclosure. [Figure 9] FIG. 1 illustrates a fusion of latent information according to one embodiment of the present disclosure. [Figure 10] FIG. 1 illustrates a fusion image according to an embodiment of the present disclosure. [Figure 11] FIG. 13 illustrates the fusion of latent information through genetic manipulation according to one embodiment of the present disclosure. [Figure 12] FIG. 13 illustrates latent information fusion according to another embodiment of the present disclosure. [Figure 13] FIG. 13 illustrates latent information fusion according to another embodiment of the present disclosure. [Figure 14] 1 is a flowchart illustrating an image generation process according to an embodiment of the present disclosure. [Figure 15]1 is a flowchart illustrating a learning process using a generative adversarial network according to an embodiment of the present disclosure. [Figure 16] FIG. 13 is a schematic diagram illustrating an image generation process according to another embodiment of the present disclosure. [Figure 17] 13 is a flowchart illustrating an image generation process according to another embodiment of the present disclosure. [Figure 18] FIG. 1 is a schematic diagram illustrating an image generation system according to an embodiment of the present disclosure. [Figure 19] FIG. 2 is a block diagram showing hardware configurations of an image generating device, a training device, and a user device according to an embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. In the following embodiments, an image generating device and a method for generating a digital image are disclosed. [Summary of the Disclosure] An image generating device according to an embodiment of the present disclosure generates fused latent information by fusing latent information (e.g., genetic code, attributes, etc.) of two images to be fused using a predetermined operation (e.g., genetic operation, etc.), and generates a fused image by fusing the two images by inputting the generated fused latent information into a trained generative model.
[0010] Specifically, a character image as shown in Fig. 1 is characterized by latent information including code (e.g., genetic code, which may be called noise) and attributes (e.g., hairstyle, hair color, eye color, skin color, facial expression, accessories such as glasses and hats, etc. All or some of them may be used.) For example, an image generating device according to the following embodiment fuses latent information of two character images to be fused as shown in Figs. 2 and 3, inputs the fused latent information to a trained generative model (e.g., a machine learning model such as a generator trained by generative adversarial networks (GANs)), and generates a fused image as shown in Fig. 4.
[0011] According to the present disclosure, the image generating device can generate a unique fusion image with plenty of variation, not just a combination of parts of the input images, by inheriting the codes and attributes of the two latent information. Also, by using a generative model, it is possible to generate a fusion image that is harmonious as a whole. [Image generation device] First, an image generating device according to an embodiment of the present disclosure will be described with reference to Figures 5 to 11. Figure 5 is a schematic diagram showing an image generating device according to an embodiment of the present disclosure.
[0012] 5, the image generating device 100 has a trained generative model, and when it acquires latent information of two images to be fused, it fuses the acquired latent information according to a predetermined operation to generate fused latent information. Then, the image generating device 100 inputs the generated fused latent information into the generative model to generate a fused image.
[0013] Specifically, the image generating device 100 may be for realizing a platform for image generation, and may provide the platform via a website, for example. For example, when a user who logs in to the platform specifies two images to be fused, the image generating device 100 acquires latent information of the two images specified by the user, and fuses the acquired latent information according to a predetermined operation. For example, the operation may be a genetic operation such as crossover, mutation, or selection on the latent information, or a predetermined synthesis operation such as arithmetic operations or logical operations. Then, the image generating device 100 may generate a fusion image from the fusion latent information using a trained generative model, and provide the generated fusion image to a user on the platform.
[0014] In the illustrated embodiment, the trained generative model is provided in the image generating device 100, but the present disclosure is not limited thereto, and the trained generative model may be disposed, for example, in an external device communicatively connected to the image generating device 100. In this case, the image generating device 100 may transmit the fusion potential information to the external device and obtain the fusion image generated by the trained generative model from the external device.
[0015] 6 is a block diagram showing a functional configuration of an image generating device according to an embodiment of the present disclosure. As shown in FIG. 6, the image generating device 100 includes a latent information acquiring unit 110, a latent information fusion unit 120, and a fusion image generating unit .
[0016] The latent information acquisition unit 110 acquires latent information of each of the two images to be fused. For example, the image to be fused may be a bitmap image such as an avatar image or a character image, and is associated with latent information that characterizes an object such as the avatar or character. Here, the latent information may be information possessed by a latent variable inferred from observed data (e.g., image data) through a model or the like.
[0017] Specifically, the latent information may include a code and an attribute, where the code may represent the inherent characteristics of the object (e.g., the bone structure of the body, the shape of the facial features, etc.), and the attribute may represent the variable characteristics of the object (e.g., the hairstyle, the facial expression, etc.). For example, as shown in Figures 7 and 8, when two images (#596506, #529690) are selected by a user as images to be fused through a user interface that displays a code displayed by a barcode, various attributes, and sub-attributes (i.e., attributes that are not expressed in the image but are inherent to the image) in association with an image of an object, the latent information acquisition unit 110 acquires the codes (in the illustrated specific example, shown as (Xc1, Xc2,..., Xc(n-1), Xcn) and (Yc1, Yc2,..., Yc(n-1), Ycn)) and attributes (in the illustrated specific example, shown as (Xa1, Xa2,..., Xa(m-1), Xam) and (Ya1, Ya2,..., Ya(m-1), Yam)) associated with these two images, and passes the latent information of each image consisting of the acquired codes and attributes to the latent information fusion unit 120.
[0018] The latent information fusion unit 120 fuses the latent information of the two images to generate fused latent information. For example, when the codes (Xc1, Xc2, . . ., Xc(n-1), Xcn) and (Yc1, Yc2, . . ., Yc(n-1), Ycn) and the attributes (Xa1, Xa2, . . ., Xa(m-1), Xam) and (Ya1, Ya2, . . ., Ya(m-1), Yam) are acquired from the latent information acquisition unit 110, the latent information fusion unit 120 generates fused latent information by fusing the acquired codes (Xc1, Xc2, . . ., Xc(n-1), Xcn) and (Yc1, Yc2, . . ., Yc(n-1), Ycn) and the attributes (Xa1, Xa2, . . ., Xa(m-1), Xam) and (Ya1, Ya2, . . ., Ya(m-1), Yam) as shown in FIG. A predetermined fusion operation is performed on the attributes (Xa1, Xa2, ···, Xa(m-1), Xam) and (Ya1, Ya2, ···, Ya(m-1), Yam) to generate fused latent information consisting of (Zc1, Zc2, ···, Zc(n-1), Zcn) and (Za1, Za2, ···, Za(m-1), Zam).
[0019] Here, the fusion operation may be performed on the code unit and attribute unit as shown in the figure, or may be performed on the latent information unit in other embodiments. Then, the latent information fusion unit 120 passes the generated fused latent information to the fused image generation unit 130. Also, the images to be fused do not necessarily have to be two different images, but may be the same image or three or more different images.
[0020] The fusion image generating unit 130 inputs the fusion latent information to the trained generative model to generate a fusion image. For example, when the fusion latent information (Zc1, Zc2, . . ., Zc(n-1), Zcn) and (Za1, Za2, . . ., Za(m-1), Zam) is acquired from the latent information fusion unit 120, the fusion image generating unit 130 inputs the acquired fusion latent information to the trained generative model to generate a fusion image as shown in FIG. 10 as an output. The generated fusion image is associated with latent information consisting of a code (Zc1, Zc2, . . ., Zc(n-1), Zcn) and an attribute (Za1, Za2, . . ., Za(m-1), Zam) as shown in the figure. In addition, the fusion image may be displayed in association with images of two origins in a parent-child relationship. Furthermore, an ancestor image of the fusion image may be displayed in a family tree format.
[0021] In one embodiment, the trained generative model may be a generator trained according to a generative adversarial network (GANs). For example, the training device 200 may train a generator and a classifier realized by a neural network or the like in advance as a generative adversarial network and provide the trained generator and classifier to the image generating device 100.
[0022] In an example of training processing, a data set of training images is provided to the training device 200, and the training device 200 first inputs a code such as a random number into a generator and obtains an image as an output. Next, the training device 200 inputs either an image generated by the generator or a training image of the data set into a discriminator, and obtains as an output a discrimination result indicating whether the input image is an image generated by the generator or a training image. Then, the training device 200 updates the parameters of the discriminator so that the discriminator makes a correct discrimination. Furthermore, the training device 200 updates the parameters of the generator so that the discriminator makes an incorrect discrimination. When a predetermined termination condition is satisfied, such as completion of execution of the above-mentioned process for a predetermined number of input data, the training device 200 provides the finally obtained generator to the image generating device 100 as a trained generative model.
[0023] In another example of training processing, a data set of training images with attributes is provided to the training device 200, and the training device 200 inputs a code such as a random number and the attributes to a generator and obtains an image as an output. Next, the training device 200 inputs either an image generated by the generator or a training image of the data set to a discriminator, and obtains as an output a discrimination result indicating whether the input image is an image generated by the generator or a training image, and what the attributes are. Then, the training device 200 updates the parameters of the discriminator so that the discriminator makes a correct discrimination. Furthermore, the training device 200 updates the parameters of the generator so that the discriminator makes an incorrect discrimination. When a predetermined termination condition is satisfied, such as completion of execution of the above-mentioned process for a predetermined number of input data, the training device 200 provides the finally obtained generator to the image generating device 100 as a trained generative model.
[0024] A trained generative model according to the present disclosure is not limited to a generator or neural network trained according to the generative adversarial network described above, but may be any type of machine learning model trained according to any other suitable training technique.
[0025] In one embodiment, the latent information fusion unit 120 may generate fused latent information by performing a genetic operation on the two pieces of acquired latent information. For example, as shown in FIG. 11, the latent information fusion unit 120 may generate a fused code and a fused attribute by crossover two codes and two attributes, respectively. In the illustrated specific example, the code Xc of the latent information X to be fused is composed of "0.2", "1.4", "0.4", ···, the attribute Xa is composed of "blonde", "with glasses", "smiling", ···, the code Yc of the latent information Y to be fused is composed of "-0.2", "0.9", "0.8", ···, and the attribute Ya is composed of "green hair", "without glasses", "blushing", ···.
[0026] At this time, the latent information fusion unit 120 may generate the code Zc "-0.2", "1.4", "0.4", ... of the fused latent information Z by crossing the elements of the codes Xc and Yc, as shown in the figure. Also, the latent information fusion unit 120 may generate the attribute Za "green hair", "wears glasses", "smiling", ... by crossing the elements of the attributes Xa and Ya, as shown in the figure. Here, the crossing described above is on an element-by-element basis, but the crossing in this embodiment is not limited to this and may be on a combination of any number of elements.
[0027] The genetic operation according to this embodiment is not limited to crossover, and may include other genetic operations such as mutation and selection. For example, in mutation, the latent information fusion unit 120 may set a value different from the elements of the latent information X and Y to be fused to one or more elements of the fused latent information Z. By introducing mutation into the fusion operation, the variation of the generated fused latent information can be increased.
[0028] In another embodiment, the latent information fusion unit 120 may generate the fused latent information by taking the average of the two acquired latent information. For example, for elements of latent information represented by numerical values such as "0.2" and "-0.2" as shown in FIG. 1, the corresponding element of the fused latent information may be determined by the average value of the corresponding elements of the two images. In this case, for elements having alternative values such as "Glasses" and "No Glasses" (such as the presence or absence of accessories), the corresponding element of the fused latent information may be randomly determined to be one of the values.
[0029] In still another embodiment, the latent information fusion unit 120 may generate fused latent information by performing a logical operation and an arithmetic operation on the two acquired latent information. The logical operation may be, for example, a logical sum, a logical product, an exclusive logical sum, or a negation operation. The arithmetic operation may be, for example, addition, subtraction, multiplication, or division. For example, a combination of the arithmetic operations such as an arithmetic mean and a harmonic mean may be included. The four arithmetic operations may be performed after applying a transformation (for example, an exponential or logarithmic operation) or a variation (for example, random noise) to the two latent information, or the four arithmetic operations may be performed on the two latent information before the transformation or variation (for example, a geometric mean or an exponential mean).
[0030] In still another embodiment, the latent information fusion unit 120 may generate fused latent information by performing a combination of one or more of the above-mentioned operations on the two acquired latent information.
[0031] In one embodiment, the latent information may further include sub-attributes. The above-mentioned attributes (called main attributes) are attributes that have a high probability of appearing in the fusion image, while the sub-attributes are attributes that are not appearing in the image and may appear in the fusion image, but have a low probability of appearing. For example, the sub-attributes may have the same content as the main attributes (e.g., hairstyle, facial expression, etc.). Either the main attribute or the sub-attributes may be probabilistically selected and used to generate the fusion latent information. As shown in Figures 7, 8 and 10, the sub-attributes may also be displayed on the user interface.
[0032] 12A, for example, the latent information X and Y of two origins are respectively composed of a code Xc, a main attribute Xa_main, and a sub-attribute Xa_sub, and a code Yc, a main attribute Ya_main, and a sub-attribute Ya_sub. Here, the probability Pmain that the main attribute is selected and the probability Psub that the sub-attribute is selected are preset (without being limited thereto, in the illustrated specific example, Pmain=75% and Psub=25%), and the attributes used to generate the fusion latent information may be determined according to the set probabilities.
[0033] For example, in the specific example shown in Fig. 12B, the main attribute Xa_main is selected for the latent information X, and the sub-attribute Ya_sub is selected for the latent information Y. Here, the probability value when selecting an attribute may be set to a different value for each element constituting the attribute. Also, depending on the set probability value, either the main attribute or the sub-attribute may be selected for each element constituting the attribute.
[0034] The latent information fusion unit 120 generates fused latent information Z as shown in FIG. 12C based on the selected latent information X (=Xc+Xa_main) and latent information Y (=Yc+Ya_sub).
[0035] As shown in the figure, the fusion latent information Z may also have a main attribute Za_main and a sub-attribute Za_sub. For example, when crossing the attributes Xa_main and Ya_sub to be fused, the latent information fusion unit 120 may construct the sub-attribute Za_sub from an element that is not selected as the main attribute Za_main of the fusion latent information Z by the crossing.
[0036] Here, when the fusion latent information Z has a code Zc, a main attribute Za_main, and a sub-attribute Za_sub, the generative model may be trained in advance so that the input main attribute Za_main is weighted higher than the sub-attribute Za_sub. The main attribute and the sub-attribute may be fused with a weighting and input to the trained generative model. Alternatively, the fusion image generating unit 130 may generate a fusion image based on the code Zc and the main attribute Za_main without inputting the sub-attribute Za_sub to the generative model.
[0037] By introducing the sub-attributes in this way, it is possible to increase the variation of the generated fusion latent information, and therefore the variation of the fusion image. The number of sub-attributes may be selected arbitrarily.
[0038] Similarly, the latent information may further include a sub-code. The above-mentioned code (called the main code) is a code that has a high probability of appearing in the fusion image. On the other hand, the sub-code is a code that has not appeared in the image and has a low probability of appearing but may appear in the fusion image. The sub-code may also be displayed on the user interface by a barcode or the like.
[0039] For example, as shown in Fig. 13A, the latent information X and Y of two origins are respectively composed of a main code Xc_main, a sub-code Xc_sub, a main attribute Xa_main and a sub-attribute Xa_sub, and a main code Yc_main, a sub-code Yc_sub, a main attribute Ya_main and a sub-attribute Ya_sub. Here, the probability Pmain that the main code is selected and the probability Psub that the sub-code is selected are preset (without being limited, in the illustrated specific example, Pmain=75% and Psub=25%), and the code used to generate the fusion latent information may be determined according to the set probabilities.
[0040] For example, in the specific example shown in Fig. 13B, the sub-code Xc_sub is selected for the latent information X, and the main code Yc_main is selected for the latent information Y. Here, the probability value for selecting a code may be set to a different value for each element constituting the code. Also, depending on the set probability value, either the main code or the sub-code may be selected for each element constituting the code.
[0041] The latent information fusion unit 120 generates fused latent information Z as shown in FIG. 13C based on the selected latent information X (=Xc_sub+Xa_main) and latent information Y (=Yc_main+Ya_sub).
[0042] As shown in the figure, the fused latent information Z may also have a main code Zc_main and a sub-code Zc_sub. For example, when crossing the codes Xc_sub and Yc_main to be fused, the latent information fusion unit 120 may construct the sub-code Zc_sub from the elements that were not selected as the main code Zc_main of the fused latent information Z by the crossing.
[0043] As with the main attribute Za_main and the sub-attribute Za_sub, the generative model may be pre-trained to weight the input main code Zc_main higher than the sub-code Zc_sub. The main code and the sub-code may be fused with a weighting and input to the trained generative model. Alternatively, the fusion image generating unit 130 may generate a fusion image based on the main code Zc and the main attribute Za_main without inputting the sub-code Zc_sub to the generative model.
[0044] By introducing subcodes in this way, the variation of the generated fusion latent information can be increased, and the variation of the fusion image can be increased. [Image generation and learning processing] Next, an image generation process and a learning process according to an embodiment of the present disclosure will be described with reference to Figs. 14 and 15. Fig. 14 is a flowchart showing an image generation process according to an embodiment of the present disclosure. The image generation process can be executed by the image generating device 100, particularly by a processor of the image generating device 100.
[0045] 14, in step S101, the latent information acquisition unit 110 acquires latent information of two origins to be merged. Typically, the latent information of these origins is held by a user on a platform, and when the user specifies an image of an origin to be merged, the latent information acquisition unit 110 acquires the latent information of the specified origin from the platform and passes it to the latent information fusion unit 120.
[0046] In step S102, the latent information fusion unit 120 fuses the acquired latent information. Specifically, the latent information fusion unit 120 may generate fused latent information by performing a genetic operation such as crossover on the acquired latent information.
[0047] In step S103, the fusion image generating unit 130 inputs the acquired fusion latent information into a trained generative model. The generative model is trained in advance by the training device 200, and may be, for example, a generator in a generative adversarial network. The trained generative model is trained to generate an image from input latent information.
[0048] In step S104, the fusion image generator 130 generates a fusion image from the trained generative model. The generated fusion image is stored in a user's folder on the platform together with latent information related to the fusion image, i.e., the fusion latent information input to the trained generative model.
[0049] 15 is a flowchart showing a training process using a generative adversarial network according to an embodiment of the present disclosure. The training process is performed by the training device 200, particularly the processor of the training device 200, for a generator and a classifier using the generative adversarial network. In the training process, a dataset of training images is provided to the training device 200.
[0050] As shown in FIG. 15, in step S201, the training device 200 inputs latent information consisting of a code (noise) such as a random number and random attributes to a generator, and obtains an image as an output.
[0051] In step S202, the training device 200 inputs the acquired image or the training image of the dataset to the classifier, and obtains as an output a classification result indicating whether the input image is an image generated by the generator or a training image. Here, attributes may be input to the classifier, or attributes may be output as a classification result.
[0052] In step S203, the training device 200 adjusts the parameters of the generator and the classifier based on the classification result according to a parameter update procedure by the generative adversarial network. Specifically, the training device 200 updates the parameters of the classifier so that the correct classification result is output, and updates the parameters of the generator so that the classifier makes an incorrect classification.
[0053] In step S204, the training device 200 judges whether the termination condition is satisfied, and if the termination condition is satisfied (S204: YES), the training device 200 terminates the learning process, and if the termination condition is not satisfied (S204: NO), the training device 200 returns to step S201 and repeats the above-mentioned steps S201 to S204 for the next training data. For example, the termination condition may be that the learning process is executed for a predetermined number of input data.
[0054] The training device 200 provides the generator obtained after the execution of the learning process to the image generating device 100 as a trained generative model. [Blockchain Compliant Code] Next, an image generation process according to another embodiment of the present disclosure will be described with reference to Figures 16 and 17. Figure 16 is a schematic diagram showing an image generation process according to another embodiment of the present disclosure. In this embodiment, the latent information is provided by a code that complies with the blockchain, which allows the latent information to be exchanged between users through a blockchain-based distribution network.
[0055] As shown in FIG. 16, the latent information may be represented by a blockchain-compliant code that conforms to any standard of a blockchain-based distribution network, such as Ethereum.
[0056] For example, in Ethereum, latent information is represented in 320 bits, specifically, code is represented in 256 bits, and attributes are represented in 64 bits, i.e., compared to the typical amount of data (e.g., 0.65KB) of latent information not represented in blockchain-compliant code, blockchain-compliant code represents latent information with a smaller amount of data.
[0057] A correspondence relationship may be defined between latent information represented by blockchain-compliant code and elements of latent information not represented by blockchain-compliant code, so that the relationship between the two can be uniquely determined. For example, the image generating device 100 may use latent information based on blockchain-compliant code purchased from another user and latent information owned by the image generating device 100 to generate a fusion image from these two pieces of latent information.
[0058] In addition, the blockchain-compliant code is not limited to the Ethereum-compliant code described above, but may be any other code that complies with a standard based on a suitable blockchain technology. Throughout this specification, the term latent information includes not only latent information that is not expressed in blockchain-compliant code, but also latent information that is expressed in blockchain-compliant code in this embodiment.
[0059] 17 is a flow chart illustrating an image generation process according to another embodiment of the present disclosure, the image generation process being executable by the image generation device 100, and in particular by a processor of the image generation device 100.
[0060] 17, in step S301, the latent information acquisition unit 110 acquires a blockchain-compliant code indicating latent information of two origins to be merged. For example, the blockchain-compliant code may be acquired from another user via a blockchain-based platform. Here, a process may be executed to convert latent information not represented by the blockchain-compliant code into latent information of the blockchain-compliant code.
[0061] In step S302, the latent information fusion unit 120 fuses the acquired blockchain-compliant codes. For example, the latent information fusion unit 120 may generate a fused blockchain-compliant code by performing a genetic operation such as crossover on the acquired blockchain-compliant codes. Here, the fusion process (including selection of sub / main, etc.) described in the case where the blockchain-compliant code is not used with reference to FIGS. 5 to 14 may be applied to the fusion process for the blockchain-compliant code. The latent information represented by the blockchain-compliant code may include at least one of a sub-attribute and a sub-code.
[0062] In step S303, the latent information fusion unit 120 converts the acquired fused blockchain-compliant code into fused latent information, which may be performed, for example, according to correspondence information indicating a correspondence relationship between the blockchain-compliant code and elements of the latent information not represented by the blockchain-compliant code.
[0063] In step S304, the fusion image generating unit 130 inputs the acquired fusion latent information into a trained generative model. The generative model is trained in advance by the training device 200, and may be, for example, a generator in a generative adversarial network. The trained generative model is trained to generate an image from the input latent information.
[0064] In step S305, the fusion image generator 130 generates a fusion image from the trained generative model. The generated fusion image may be stored in a user's folder on the platform together with the fusion potential information.
[0065] In this embodiment, the fusion process was performed on latent information represented by blockchain-compliant code, but the present disclosure is not limited to this, and the blockchain-compliant code to be fused may be converted into latent information that is not blockchain-compliant code, and the fusion process may be performed on the converted latent information.
[0066] In this way, each user can have the latent information represented by the blockchain-compliant code, which is less data-intensive, instead of the latent information not represented by the blockchain-compliant code, which is relatively large. In addition, the blockchain and smart contracts can ensure the value and ownership of digital assets. By utilizing the properties of generative models, the fusion of two images is possible through smart contracts, and the result of the fusion is traceable and unpredictable, which makes the game more interesting for users.
[0067] Next, an image generating system according to an embodiment of the present disclosure will be described with reference to Fig. 18. In this embodiment, a part or all of the functions of the image generating device 100 may be provided as a cloud service in the image generating system 10, and a user may generate a fusion image and obtain the generated fusion image via a user device 300 such as a personal computer. For example, the image generating device 100 may be installed in a different location from the user device 300.
[0068] As shown in FIG. 18, the image generation system 10 includes an image generation device 100 and a user device 300 communicatively connected to the image generation device 100 .
[0069] When the user device 300 instructs the image generating device 100 via the network to generate images to be fused, the image generating device 100 generates fused latent information by fusing the latent information of the images to be fused based on the received instruction. The image generating device 100 then inputs the generated fused latent information into a trained generative model to generate a fused image, and transmits the generated fused image to the user device 300. The user device 300 acquires the fused image generated by the image generating device 100 via the network. Note that the latent information to be fused instructed by the user may be latent information represented by a blockchain-compliant code, or may be latent information not represented by a blockchain-compliant code.
[0070] 18, the latent information to be merged is transmitted from the user device 300 to the image generating device 100, but information on two origin images specified by the user may be transmitted from the user device 300 to the image generating device 100. In this case, the image generating device 100 may acquire the latent information to be merged based on an instruction from the user.
[0071] Although the above-described embodiment focuses on image generation, the present disclosure is not limited thereto and may be applied to the generation of any type of information that can be generated from latent information, such as video, audio, etc. Furthermore, the above-described image generation technology may be used in games that generate new characters, etc. [Hardware configuration of image generation device, training device, and user device] In the image generating device 100, the training device 200, and the user device 300 in the embodiments, each function may be a circuit configured with an analog circuit, a digital circuit, or an analog-digital mixed circuit. Also, each function may be provided with a control circuit. Each circuit may be implemented by an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or the like.
[0072] In all of the above descriptions, at least a part of the image generating device 100, the training device 200, and the user device 300 may be configured with hardware, or may be configured with software, and a CPU (Central Processing Unit) or the like may perform information processing by the software. In the case of being configured with software, the image generating device 100, the training device 200, the user device 300, and a program that realizes at least a part of the functions thereof may be stored in a storage medium and read and executed by a computer. The storage medium is not limited to a removable medium such as a magnetic disk (e.g., a flexible disk) or an optical disk (e.g., a CD-ROM or a DVD-ROM), but may be a fixed storage medium such as a hard disk device or an SSD (Solid State Drive) that uses a memory. In other words, the information processing by the software may be specifically implemented using hardware resources. Furthermore, the processing by the software may be implemented in a circuit such as an FPGA and executed by hardware. The execution of a job may be performed using an accelerator such as a GPU (Graphics Processing Unit).
[0073] For example, the computer can become the device of the above embodiment by reading out the dedicated software stored in a computer-readable storage medium. The type of storage medium is not particularly limited. Also, the computer can become the device of the above embodiment by installing the dedicated software downloaded via a communication network. In this way, the information processing by the software is specifically implemented using hardware resources.
[0074] 19 is a block diagram showing an example of a hardware configuration according to an embodiment of the present disclosure. The image generating device 100, the training device 200, and the user device 300 each include a processor 101, a main memory device 102, an auxiliary memory device 103, a network interface 104, and a device interface 105, and can be realized as a computer device in which these are connected via a bus 106.
[0075] 14 includes one of each of the image generating device 100, the training device 200, and the user device 300, but may include multiple of the same components. Also, although one image generating device 100, training device 200, and user device 300 are shown, the software may be installed in multiple computer devices, and each of the multiple image generating devices 100, training devices 200, and user devices 300 may execute different parts of the processing of the software. In this case, each of the multiple image generating devices 100, training devices 200, and user devices 300 may communicate with each other via a network interface 104, etc.
[0076] The processor 101 is an electronic circuit (processing circuit, processing circuitry) including a control unit and a calculation device of the image generating device 100, the training device 200, and the user device 300. The processor 101 performs calculation processing based on data and programs input from each device in the internal configuration of the image generating device 100, the training device 200, and the user device 300, and outputs the calculation result and a control signal to each device. Specifically, the processor 101 controls each component constituting the image generating device 100, the training device 200, and the user device 300 by executing an OS (Operating System) and an application of the image generating device 100, the training device 200, and the user device 300. The processor 101 is not particularly limited as long as it can perform the above processing. The image generating device 100, the training device 200, the user device 300, and each of their components are realized by the processor 101. Here, the processing circuit may refer to one or more electrical circuits disposed on a single chip, or to one or more electrical circuits disposed on two or more chips or devices. When multiple electronic circuits are used, the electronic circuits may communicate with each other by wire or wirelessly.
[0077] The main memory 102 is a memory device that stores instructions executed by the processor 101 and various data, and information stored in the main memory 102 is directly read by the processor 101. The auxiliary memory 103 is a memory device other than the main memory 102. These storage devices refer to any electronic components capable of storing electronic information, and may be memory or storage. In addition, the memory may be either a volatile memory or a non-volatile memory. A memory for storing various data in the image generating device 100, the training device 200, and the user device 300, for example, a memory, may be realized by the main memory 102 or the auxiliary memory 103. For example, at least a part of the memory may be implemented in the main memory 102 or the auxiliary memory 103. As another example, when an accelerator is provided, at least a part of the memory described above may be implemented in a memory provided in the accelerator.
[0078] The network interface 104 is an interface for connecting to the communication network 108 wirelessly or by wire. The network interface 104 may be one that conforms to an existing communication standard. The network interface 104 may exchange information with an external device 109A that is connected for communication via the communication network 108.
[0079] The external device 109A includes, for example, a camera, a motion capture device, an output destination device, an external sensor, an input source device, etc. The external device 109A may also be a device having some of the functions of the components of the image generation device 100, the training device 200, and the user device 300. The image generation device 100, the training device 200, and the user device 300 may receive some of the processing results of the image generation device 100, the training device 200, and the user device 300 via the communication network 108 like a cloud service.
[0080] The device interface 105 is an interface such as a Universal Serial Bus (USB) that directly connects to the external device 109B. The external device 109B may be an external storage medium or a storage device. The memory may be realized by the external device 109B.
[0081] The external device 109B may be an output device. The output device may be, for example, a display device for displaying an image, or a device for outputting sound or the like. Examples of the output device include, but are not limited to, an LCD (Liquid Crystal Display), a CRT (Cathode Ray Tube), a PDP (Plasma Display Panel), an organic EL (ElectroLuminescence) display, and a speaker.
[0082] The external device 109B may be an input device. The input device includes devices such as a keyboard, a mouse, a touch panel, and a microphone, and provides information input by these devices to the image generating device 100, the training device 200, and the user device 300. Signals from the input device are output to the processor 101.
[0083] For example, the latent information acquisition unit 110, the latent information fusion unit 120, the fusion image generation unit 130, and the like of the image generation device 100 in this embodiment may be realized by one or more processors 101. Furthermore, the memories of the image generation device 100, the training device 200, and the user device 300 may be realized by the main storage device 102 or the auxiliary storage device 103. Furthermore, the control device 100 may be equipped with one or more memories.
[0084] In this specification, "at least one of a, b, and c" is an expression that includes not only combinations of a, b, c, ab, ac, bc, and abc, but also multiple combinations of the same elements such as aa, abb, and aabbcc. It is also an expression that covers configurations that include elements other than a, b, and c, such as the combination abcd.
[0085] Similarly, in this specification, "at least one of a, b, and c" is an expression that includes not only combinations of a, b, c, ab, ac, bc, and abc, but also multiple combinations of the same elements such as aa, abb, and aabbcc. It is also an expression that covers configurations that include elements other than a, b, and c, such as the combination abcd.
[0086] Although the examples of the present disclosure have been described in detail above, the present disclosure is not limited to the specific embodiments described above, and various modifications and variations are possible within the scope of the gist of the present disclosure described in the claims. [Explanation of symbols]
[0087] 10 Image Generation System 100 Image generating device 101 Processor 102 Main storage 103 Auxiliary storage device 104 Network Interface 105 Device Interfaces 108 Communication Network 109A,B External device 110 Latent information acquisition unit 120 Latent information fusion department 130 Fusion image generation unit 200 Training equipment 300 User device
Claims
1. at least one processor generates data by inputting the latent information into a generative model; The latent information is information represented by a blockchain-compliant code. Data generation method.
2. At least one processor converts latent information that is blockchain-compliant code into latent information that is not blockchain-compliant code, and generates data by inputting the converted latent information into a generative model. Data generation method.
3. The at least one processor converts the latent information, which is a blockchain-compliant code, into the converted latent information, which is not a blockchain-compliant code, based on a predetermined correspondence relationship. The data generating method according to claim 2 .
4. The predetermined correspondence relationship is a correspondence relationship between an element of the latent information that is a blockchain-compliant code and an element of the converted latent information that is not a blockchain-compliant code. The data generating method according to claim 3 .
5. The latent information, which is a blockchain-compliant code, has a smaller amount of data than the converted latent information, which is not a blockchain-compliant code; The data generating method according to any one of claims 2 to 4.
6. The latent information includes information characterizing an object. The data generating method according to any one of claims 1 to 5.
7. The latent information includes at least one of information regarding the object's hairstyle, hair color, eye color, skin color, facial expression, and the presence or absence of an accessory. The data generating method according to claim 6.
8. The latent information includes inherent and variable features of the object. The data generating method according to claim 6.
9. The latent information is fused latent information generated by fusing the first latent information and the second latent information. The data generating method according to any one of claims 1 to 8.
10. The at least one processor generates the fused latent information by fusing the first latent information and the second latent information. The data generating method according to claim 9.
11. The at least one processor generates the fused latent information by performing a genetic operation using the first latent information and the second latent information. The data generating method according to claim 10.
12. The genetic operation includes at least one of crossover, mutation, and selection. The data generating method according to claim 11.
13. The first potential information includes a main attribute and a sub-attribute, The second latent information includes a main attribute and a sub attribute. The data generating method according to claim 11 or 12.
14. The at least one processor generates the fusion latent information by performing at least one of four arithmetic operations and logical operations using the first latent information and the second latent information. The data generating method according to any one of claims 9 to 13.
15. the at least one processor generates the fused latent information using random information; The data generating method according to any one of claims 9 to 14.
16. The first latent information is information expressed in a blockchain-compliant code. The data generating method according to any one of claims 9 to 15.
17. The first potential information is information about a purchase made by a second user from a first user. The data generating method according to any one of claims 9 to 16.
18. The first latent information includes information characterizing a first object; the second latent information includes information characterizing a second object; The data generating method according to any one of claims 9 to 17.
19. The first latent information includes at least one of information regarding the hairstyle, hair color, eye color, skin color, facial expression, and the presence or absence of an accessory of the first object; The second latent information includes at least one of information regarding a hairstyle, a hair color, an eye color, a skin color, a facial expression, and the presence or absence of an accessory of the second object.
20. The data generating method according to claim 18.
20. The first latent information includes a unique feature and a variable feature of the first object; the second latent information includes an inherent feature and a variable feature of the second object; 20. The data generating method according to claim 18.
21. the at least one processor displays the object, the first object, and the second object in a family tree format on a display device. A data generating method according to any one of claims 18 to 20 dependent on claim 6.
22. The generative model is a neural network.
22. The data generating method according to any one of claims 1 to 21.
23. The data is at least one of an image, a video, or an audio.
23. The data generating method according to any one of claims 1 to 22.
24. at least one processor, The at least one processor is adapted to execute a method for generating data according to any one of claims 1 to 23. Data generation device.
25. Causing at least one processor to carry out the data generation method according to any one of claims 1 to 23. program.
Citation Information
Patent Citations
Computer system and game system
JP2018171271A
Information provision device, information provision system, information provision method and program
JP2018173711A
Management device, management method, and program
JP2019003402A
