Program, information processing method, information processing device, and manufacturing method

Abstract

A program according to one aspect of the present embodiment causes a computer to execute processing for: acquiring a user image obtained by imaging a user; acquiring content serving as a target; and outputting a plurality of mutually different types of generated images generated on the basis of the user image and the content. In addition, in the program according to one aspect of the present embodiment, the content refers to a character image, and the generated images are output by means of an image generation model that uses the user image and a character image to output such images.

Description

Program, information processing method, information processing device and manufacturing method

[0001] The present technology relates to a program, an information processing method, and an information processing device.

[0002] Conventionally, a technique for displaying an avatar of a user in a virtual space has been proposed (for example, Patent Document 1).

[0003] Japanese Patent Application Laid-Open No. 2024-021529

[0004] When a user's image is displayed in a virtual space that imitates the world of a work containing content, such as a particular anime or game, there has been a problem in that the user's image may not match the worldview of the work.

[0005] The present disclosure has been made in consideration of the above circumstances, and aims to provide a program or the like that can output a user's image that matches the worldview of a work that includes content.

[0006] A program according to one embodiment of the present disclosure causes a computer to execute a process of acquiring a user image of a user, acquiring target content, and outputting multiple types of mutually different generated images generated based on the user image and the content.

[0007] In a program according to an embodiment of the present disclosure, an image of a user that matches the worldview of a work can be output.

[0008] 1 is an explanatory diagram showing an overview of an information processing device; FIG. 1 is a block diagram showing an example of a configuration example of an information processing device; FIG. 2 is an explanatory diagram showing an example of image conversion specification information (prompt) settings; FIG. 3 is an explanatory diagram showing an example of an edge detection module; FIG. 4 is an explanatory diagram showing an example of a method for learning an image generation model; FIG. 5 is an explanatory diagram showing an example of an image generation model; FIG. 6 is an explanatory diagram showing an example of a work table; FIG. 7 is an explanatory diagram showing an example of a character table; FIG. 8 is an explanatory diagram showing an example of a body image table; FIG. 9 is an explanatory diagram showing an example of a background image table; FIG. 10 is an explanatory diagram showing an example of a work selection screen; FIG. 11 is an explanatory diagram showing an example of a user image shooting screen; FIG. 12 is an explanatory diagram showing an example of a character image selection screen; FIG. 13 is an explanatory diagram showing an example of a work level selection screen; FIG. 14 is an explanatory diagram showing an example of a setting item selection screen; FIG. 15 is an explanatory diagram showing an example of a body selection screen; FIG. 16 is an explanatory diagram showing an example of a background selection screen; FIG. 17 is a flowchart showing an example of generated image generation processing; FIG. 18 is an explanatory diagram showing an example of an image generation model according to a second embodiment; FIG. 19 is an explanatory diagram showing an example of a character image selection screen according to the second embodiment; FIG. 20 is an explanatory diagram showing an example of a method for learning an image generation model according to a third embodiment; FIG. 21 is an explanatory diagram showing an example of a work table according to the third embodiment; FIG. 10 is an explanatory diagram showing an example of a generated image transmission screen. FIG. 11 is an explanatory diagram showing an example of a reading screen on a user terminal. FIG. 12 is an explanatory diagram showing an example of a generated image display screen on a user terminal. FIG. 13 is an explanatory diagram showing an overview of a generated image utilization system according to a fifth embodiment. FIG. 14 is an explanatory diagram showing an example of a generated image transmission screen according to the fifth embodiment. FIG. 15 is an explanatory diagram showing an example of a generated image table. FIG. 16 is an explanatory diagram showing an example of a reading screen on a user terminal according to the fifth embodiment. FIG. 17 is an explanatory diagram showing an example of an avatar generation model. FIG. 18 is an explanatory diagram showing an example of an avatar display screen. FIG. 19 is an explanatory diagram showing an example of a generated image table according to a sixth embodiment. FIG. 19 is an explanatory diagram showing an example of a virtual space display screen on a user terminal. FIG. 19 is an explanatory diagram showing an example of a similarity calculation model. FIG. 19 is an explanatory diagram showing an example of a work quality selection screen according to a seventh embodiment. FIG. 19 is an explanatory diagram showing an example of a work quality selection screen according to a modified example of the seventh embodiment. FIG. 19 is a flowchart showing an example of a generated image generation process according to a modified example of the seventh embodiment.

[0009] 1 is an explanatory diagram showing an overview of an information processing device 1. The information processing device 1 is, for example, a photo ID machine that takes an image of a user and, based on the taken image (user image), generates a generated image that resembles the style of a character image that is the target content. The information processing device 1 accepts input or selection from the user of various items for adjusting the generated image, prints out the generated image adjusted based on the input content, and provides it to the user.

[0010] 2 is a block diagram showing an example of the configuration of the information processing device 1. The information processing device 1 includes a processing unit 11, a storage unit 12, a display unit 13, an input unit 14, an image capturing unit 15, and a printing unit 16. The processing unit 11 is configured with a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphic Processing Unit), or the like, and performs various control processes, arithmetic processes, and the like by reading and executing a program P (program product) or the like stored in advance in the storage unit 12.

[0011] The storage unit 12 of the information processing device 1 is, for example, a volatile memory and a non-volatile memory. The storage unit 12 stores a program P, a prompt extraction model M1, an edge detection module M2, an image generation model M3, a work table 121, a character table 122, a body image table 123, and a background image table 124. The program P may be provided to the information processing device 1 using a computer-readable recording medium 12a. The recording medium 12a may be, for example, a portable memory. Examples of portable memory include a CD-ROM, a USB (Universal Serial Bus) memory, an SD card, a microSD card, and a CompactFlash (registered trademark) memory. When the recording medium 12a is a portable memory, the processing element of the processing unit 11 may read the program P from the recording medium 12a using a reading device (not shown). The read program P is written to the storage unit 12. Furthermore, the program P may be provided to the information processing device 1 by a communication unit (not shown) communicating with an external device. The prompt extraction model M1, edge detection module M2, image generation model M3, work table 121, character table 122, body image table 123, and background image table 124 will be described in detail below.

[0012] The display unit 13 of the information processing device 1 is configured, for example, by a liquid crystal panel, and displays various screens, which will be described later. The input unit 14 accepts input of information or selections from a user, which will be described later. In this embodiment, the display unit 13 and the input unit 14 are an operation panel of the ID photo machine, and are configured as an integrated touch panel.

[0013] The photographing unit 15 of the information processing device 1 is a camera provided in the passport photo machine for taking facial photographs. The photographing unit 15 photographs a user image showing the user's face. The printing unit 16 is a photo printer provided in the passport photo machine. The printing unit 16 prints a generated image based on the user image onto printing paper, a sticker for an acrylic stand, or the like.

[0014] FIG. 3 is an explanatory diagram showing an example of image conversion specification information (prompt) settings. The image conversion specification information (prompt) is information that specifies the conditions for a converted image when converting a user image, and includes, for example, the gender, age, hair color, eye color, accessories, and clothing of the converted image. The image conversion specification information (prompt) uses features extracted from the user image by the prompt extraction model M1. Note that the image conversion specification information (prompt) may be set by a user selecting and instructing on an operation terminal screen. The prompt extraction model M1 may be configured, for example, by a model that detects objects contained in an input image and outputs the class of the detected object, such as a convolutional neural network (CNN), a you-only look-at-once (YOLO), a vision transformer, or a single-shot multibox detector (SSD); a large language model (LLM) that verbalizes the features of the input image, such as a generative pre-trained transformer (GPT); or an image classification model such as contrastive language-image pre-training (CLIP). When the prompt extraction model M1 is configured using a neural network including a CNN, the input layer of the prompt extraction model M1 has multiple neurons that receive input pixel values ​​of a user image and passes the input pixel values ​​to the middle layer. The middle layer has multiple neurons that extract image features of the user image and passes the extracted image features to the output layer. The output layer outputs a user class (feature) contained in the user image based on the image features. In the example shown in FIG. 3 , the user classes (features) include "male," "20s," "black hair," "black eyes," "glasses," and "shirt." Note that the classes output by the prompt extraction model M1 are not limited to these, and other classes may also be output, such as facial expression, gaze, or background. Furthermore, the classes output by the prompt extraction model M1 are not limited to words, and may be sentences such as "he wears glasses and has black hair that is long enough so that his bangs do not cover his glasses."

[0015] FIG. 4 is an explanatory diagram showing an example of the edge detection module M2. The edge detection module M2 is, for example, a Canny edge detector using a Sobel filter. The edge detection module M2 may also be an edge detector using a Gaussian filter or a Laplacian filter. The edge detection module M2 has the function of detecting the boundaries (edges) between bright and dark areas of an input image based on the luminance of each pixel in the input image. Thus, when a user image is input, the edge detection module M2 detects the user's characteristic features (hair, eyes, nose, mouth, etc.) and contours, and outputs the positions of the characteristic features and contours (edge ​​information). In the output image shown in FIG. 4, the positions of the characteristic features and contours are indicated by dashed lines.

[0016] FIG. 5 is an explanatory diagram showing an example of a training method for the image generation model M3. The image generation model M3 is configured, for example, by a text-to-image model including stable diffusion. The image generation model M3 may be a diffusion model such as DALL-E3, or a generative adversarial network (GAN). The image generation model M3 is a model trained to output an image generated based on a prompt and edge information indicating the characteristics of the image to be generated when the prompt and edge information are input. When the image generation model M3 is configured using stable diffusion, the image generation model M3 is trained (pre-trained) using images obtained from copyright holders or publicly available images. The processing unit 11 of the information processing device 1 fine-tunes (additional training) the image generation model M3, which has been pre-trained by scraping, for each character using character images of characters included in the work as training data, thereby training the model to output images that resemble the artistic style of each character image. The image generation model M3 may also be fine-tuned on a server device external to the information processing device 1.

[0017] The image generation model M3 is fine-tuned, for example, by Low-Rank Adaptation (LORA). Specifically, by adding a low-rank decomposition matrix based on the character image and a caption (character image feature) extracted from the character image to each layer of the image generation model M3, the image generation model M3 is fine-tuned to output an image that resembles the style of the character image. In other words, the image generation model M3 is fine-tuned to learn the style of the character image. Fine-tuning (additional learning) of the image generation model M3 is performed using a dataset of multiple character images for each character, and multiple image generation models M3 (M3a, M3b, M3c, etc.) that have learned the style of each character are generated. Note that the image generation model M3 may also be trained using, for example, Dream Booth, Textual Inversion, Prefix Tuning, or Prompt Tuning.

[0018] The example shown in FIG. 5 illustrates an image generation model M3a fine-tuned using the character image of character A, an image generation model M3b fine-tuned using the character image of character B, and an image generation model M3c fine-tuned using the character image of character C. The training data dataset for fine-tuning the image generation model M3 includes one character image and a caption extracted from the character image. The image generation model M3 is fine-tuned by inputting a dataset related to, for example, 20 character images per character. Note that the number of datasets input to the image generation model M3 may be 19 or less or 21 or more. In the example shown in FIG. 5, one dataset per character image is illustrated, and other datasets are omitted. The storage unit 12 of the information processing device 1 stores fine-tuned (additionally learned) image generation models M3 (M3a, M3b, M3c, ...) related to each character. In other words, the character images are used as training data for the image generation model M3 to output generated images.

[0019] FIG. 6 is an explanatory diagram showing an example of the image generation model M3. As described above, the image generation model M3 is configured with either or both of an image-to-image model including stable diffusion and a text-to-image model. When a user image, a prompt, and edge information are input, the image generation model M3 outputs a generated image based on the user image from which the prompt was extracted and the character image used for fine tuning. The processing unit 11 of the information processing device 1 reads an image generation model M3 (one of M3a, M3b, M3c, etc.) related to a character selected by the user (see FIG. 13 ), and inputs the user image, the prompt (class) extracted from the user image by the prompt extraction model M1, and the edge information extracted by the edge detection module M2 into the read image generation model M3. Note that all or part of the prompt input to the image generation model M3 may be instructions input by the user. The prompt input to the image generation model M3 may also include a designation of the use of the generated image or a designation of the number of types of generated images to be output by the image generation model M3. When the image generation model M3 is a stable diffusion model, the encoder of the VAE (Variational AutoEncoder) of the image generation model M3 converts the input user image into a latent representation and passes it to the diffusion model. The text encoder of the image generation model M3 vector-converts the input prompt and passes it to the diffusion model. The diffusion model generates a pixel-by-pixel probability map of the generated image based on the latent representation of the user image, the vector of the acquired prompt, and the input edge information, and passes it to the VAE decoder. The VAE decoder constructs and outputs the generated image based on the received probability map. In other words, the user image is directly used as input data for the image generation model M3 to output the generated image. The image generation model M3 (Stable Diffusion) may be equipped with additional functions (e.g., a control net), and the processing unit 11 may input control information related to image generation, such as pose or composition specification, to the image generation model M3.

[0020] In this embodiment, the image generation model M3 outputs multiple generated images with different degrees of artistic quality. The artistic quality indicates the degree to which the generated image reflects the artistic style of the artwork and varies depending on the degree of edge information reflection (weight) when generating the generated image. The weight varies, for example, between 0 and 1.0, with higher values ​​indicating a higher degree of edge information reflection and lower values ​​indicating a lower degree of edge information reflection. The artistic quality is, for example, a value obtained by subtracting the degree of edge information reflection from 1.0. That is, the higher the degree of edge information reflection, the lower the artistic quality of the generated image, and the lower the degree of edge information reflection, the higher the artistic quality of the generated image. In the example shown in FIG. 6 , the image generation model M3 outputs four levels of edge information weights: 0.9, 0.7, 0.3, and 0.1. The processing unit 11 of the information processing device 1 may include information specifying the weight of edge information in the prompt and output a generated image in which edge information is reflected based on the specified weight. The processing unit 11 may also output multiple types of generated images with different degrees of applicability of the model learned by LoRA. In this case, the degree of work may refer to the applicability of the model trained by LoRA. Also, the image generation model M3 may generate multiple different images by changing the seed value in image generation.

[0021] The data input to the image generation model M3 may include at least one of a user image, a prompt, or edge information. For example, if the data input to the image generation model M3 includes a prompt and edge information, information extracted from the user image (prompt and edge information) is input to the image generation model M3. That is, the user image is indirectly used as input data for the image generation model M3 to output a generated image. The data input to the image generation model M3 may also include a character image of a character selected by the user (see FIG. 13 ). If the data input to the image generation model M3 includes a user image and a character image, the prompt input to the image generation model M3 may include an instruction to output a generated image by making the user image resemble the style of the character image. In this case, the character image is used as input data for the image generation model M3 to output a generated image. The data input to the image generation model M3 includes at least one of a user image, a character image, edge information, and a prompt. A user image capturing the user's entire body may also be input to the image generation model M3. At this time, the image generation model M3 may output a generated image of the entire body.

[0022] FIG. 7 is an explanatory diagram showing an example of the work table 121. The work table 121 records the titles (work names) of works such as manga, anime, movies, or games in which characters whose character images are the target of training for the image generation model M3 appear. The management items (fields) of the work table 121 include a work ID field, a work name field, a work logo field, and a character table ID field. The work ID field stores an ID assigned to the work. The work name field stores the work name. The work logo field stores the work's logo, for example, in file format. The character table ID field stores an ID for identifying the character table 122 (see FIG. 8) in which information about characters appearing in each work is stored.

[0023] FIG. 8 is an explanatory diagram showing an example of the character table 122. The character table 122 records information about characters appearing in the work. The management items (fields) of the character table 122 include a character ID field, a character name field, a character image field, and an image generation model field. The character ID field stores an ID assigned to a character. The character name field stores the character name of the character. The character image field stores, for example, an image file of one of the character images used to fine-tune the image generation model M3 related to the character. The image generation model field stores the type of image generation model M3 (one of M3a, M3b, M3c, etc.) fine-tuned by the character image of the character.

[0024] Furthermore, the character tables 122 are assigned attribute information for distinguishing between the multiple character tables 122. The attribute information includes the character table ID stored in the work table 121 and the work name read from the work table 121 using the character table ID as a key.

[0025] FIG. 9 is an explanatory diagram showing an example of the body image table 123. The body image table 123 stores, for each character, a file of a body image (see FIG. 16) to be combined with a generated image (corrected generated image). For example, an image below the neck, cropped from a character image used to train the image generation model, is used as the body image. The management items of the body image table 123 include a character ID field and multiple body image fields. The character ID field stores the character ID of the character. The body image field stores the body image of each character. The body image table 123 stores files of body images in different poses (postures) for each character.

[0026] FIG. 10 is an explanatory diagram showing an example of the background image table 124. The background image table 124 stores files of background images (see FIG. 17) to be combined with the generated image (body combination image). The management items of the background image table 124 include a theme field, a display period field, and a background image field. The theme field stores the theme (event name) of the background image. The display period field stores the date period during which the background image is displayed as an option on the background selection screen shown in FIG. 17. The background image field stores files of the background image. Note that each background image may be displayed on the background selection screen regardless of the date or the date-based theme (event name).

[0027] 11 is an explanatory diagram showing an example of a work selection screen. For example, when the processing unit 11 of the information processing device 1 receives an input from a user instructing the user to start generating a generated image, the processing unit 11 displays the work selection screen on the display unit 13. The work selection screen displays a search field, multiple work display fields, and a start shooting button. The input unit 14 receives input of a work name from the user in the search field displayed on the display unit 13 to search for a work. Based on the input content in the search field, the processing unit 11 identifies works stored in the work table 121 whose work names have a high degree of similarity to the input content.

[0028] The work display field displays the name of a work identified from among the works stored in the work table 121 (see FIG. 7 ) based on the input content in the search field, or the logo of the work. The work display field also displays a checkbox that allows the user to select the name of the work. The work display field may also display, for example, character images of characters appearing in the work. In the example shown in FIG. 11 , three works are identified based on the input content in the search field, and three work display fields are displayed on the work selection screen. However, the number of works identified based on the input content in the search field and displayed in the work display fields may be two or less, or four or more.

[0029] When one of the check boxes in the work display fields displayed on the work selection screen is checked, the start shooting button becomes pressable. When the start shooting button is pressed, the processing unit 11 of the information processing device 1 starts the shooting unit 15 and displays a user image shooting screen (see FIG. 12 ) on the display unit 13.

[0030] FIG. 12 is an explanatory diagram showing an example of a user image capture screen. When the capture start button is pressed on the work selection screen, the processing unit 11 of the information processing device 1 displays the user image capture screen on the display unit 13. A captured image field is displayed on the user image capture screen. The captured image field displays images being captured by the capture unit 15 in real time. The processing unit 11 determines, for example, the image captured by the capture unit 15 when a predetermined time (e.g., 10 seconds) has elapsed since the user image capture screen was displayed on the display unit 13 as the user image. Note that a capture button may be displayed on the user image capture screen, and the image captured when a predetermined time has elapsed since the capture button was pressed may be determined as the user image. Alternatively, the processing unit 11 may display, on the display unit 13, multiple images captured before the predetermined time has elapsed, and have the input unit 14 accept input for selecting an image to be used as the user image from the multiple images. When a user image has been captured, the processing unit 11 displays a character image selection screen on the display unit 13. In addition, the processing unit 11 may acquire, for example, a user image taken by a smartphone owned by the user or a user image stored on the smartphone via communication, and display a character image selection screen on the display unit 13.

[0031] FIG. 13 is an explanatory diagram showing an example of a character image selection screen. The character image selection screen displays character images of multiple characters appearing in the work selected on the work selection screen (see FIG. 11 ) and a generation start button. That is, the processing unit 11 of the information processing device 1 identifies the character table 122 for the selected work based on the character table ID for the work selected on the work selection screen, and displays the character images stored in the identified character table 122, as well as the character names and character IDs of each character, on the character image selection screen. Each character image is displayed with a check box that allows the user to select a character image to which the user image's style is to be applied. When any one of the check boxes for the multiple character images is checked, the generation start button becomes pressable.

[0032] When the generation start button is pressed on the character image selection screen, the processing unit 11 of the information processing device 1 reads out an image generation model M3 related to the character of the selected character image based on the character table 122, and inputs the user image, a prompt extracted from the captured user image, and edge information into the read image generation model M3. Note that the processing unit 11 may input at least one of the user image, the prompt, and the edge information into the image generation model M3. When a generated image is output by the image generation model M3, the processing unit 11 displays a work quality selection screen (see FIG. 14) on the display unit.

[0033] FIG. 14 is an explanatory diagram showing an example of a quality level selection screen. The quality level selection screen displays a captured user image, a character image selected on the character image selection screen (see FIG. 13 ), multiple types of generated images with different quality levels output by the image generation model M3, and a setting button. Note that multiple generated images with the same quality level but different image generation seed values, for example, may also be displayed on the quality level selection screen. In the example shown in FIG. 14 , each generated image is displayed with a quality level (a value obtained by subtracting the weight of the edge information from 1.0). Note that each generated image may also be displayed with the weight of the edge information itself. Furthermore, on the quality level selection screen, the user image, the character image, and multiple types of generated images in order of similarity to the user image are displayed for comparison on the same screen. The display arrangement of the user image, the character image, and multiple generated images on the quality level selection screen is not limited to that shown in FIG. 14 . For example, the processing unit 11 of the information processing device 1 may display each image in the following order: user image, generated image, and character image. Furthermore, the processing unit 11 may display generated images in such a way that generated images with higher degrees of artistic quality are placed closer to the character image, and generated images with lower degrees of artistic quality are placed closer to the user image. In this case, it becomes easier to grasp the gradual change in artistic quality from the character image to the user image, making it easier for the user to select generated images based on differences in artistic quality.

[0034] Furthermore, on the artwork quality selection screen, each generated image is displayed with a check box attached thereto for accepting the user's selection of the generated image to be printed. When any one of the check boxes for the multiple generated images is checked, a setting button becomes pressable. When the setting button is pressed, the processing unit 11 of the information processing device 1 displays a setting item selection screen (see FIG. 15 ) on the display unit 13.

[0035] FIG. 15 is an explanatory diagram showing an example of a setting item selection screen. The setting item selection screen displays the generated image selected on the quality selection screen (see FIG. 14 ), the character image selected on the character image selection screen (see FIG. 11 ), the corrected generated image, and a setting item field. The setting item field displays setting items related to the hair, eye color, and glasses of the generated image. Each setting item includes multiple options (setting values). In the example shown in FIG. 15 , the hair setting item includes options (setting values) of “red hair” and “blue hair,” the eye color setting item includes options (setting values) of “red eyes” and “blue eyes,” and the glasses setting item includes options (setting values) of “with glasses” and “without glasses.” Note that the setting items are not limited to those described above and may include options related to clothing, for example.

[0036] Each setting item option is displayed with a checkbox for accepting the selection of the option to be reflected in the corrected generated image. The user can input one option for each setting item. Before selecting a setting item option, the corrected generated image is the same as the generated image. When a setting item option (setting value) is selected, the corrected generated image is corrected according to the selected option (setting value). For example, when "with glasses" is selected for the glasses setting item, the corrected generated image is created by superimposing glasses on the generated image. Note that if glasses are already included in the generated image, selecting "without glasses" may remove the glasses from the corrected generated image. Furthermore, the processing unit 11 of the information processing device 1 may output the corrected generated image by re-inputting the content of the selected correction item option (setting value) into the image generation model M3 in a prompt. Note that in this embodiment, the processing unit 11 accepts the selection of a setting item option after accepting the selection of a generated image. However, the processing unit 11 may also cause the image generation model M3 to generate a generated image after accepting the selection of a setting item option. At this time, the prompt input to the image generation model M3 may include information related to the options (setting values) of the selected setting item.

[0037] A body selection button is displayed on the setting item selection screen. After inputting a selection option for each setting item, the user can press the body selection button. Note that it is not necessary to select at least one option for each setting item, and there may be setting items for which no option is selected (not edited in the corrected generated image). Also, it is not necessary to edit all setting items. When the body selection button is pressed, the processing unit 11 of the information processing device 1 displays a body selection screen (see FIG. 16 ) on the display unit 13.

[0038] FIG. 16 is an explanatory diagram showing an example of the body selection screen. The body selection screen displays a corrected generated image (see FIG. 15 ) edited by accepting selections of setting options. The body selection screen also displays multiple body image options read from the body image table 123. For example, an image below the neck trimmed from the character image used in training the image generation model M3 is used as the body image. The processing unit 11 reads from the body image table 123 body images related to the character used in training the image generation model M3 that output the generated image, and displays them as options on the body selection screen. Each body image is displayed with a checkbox that allows the user to select a body image to be combined with the corrected generated image. The processing unit 11 of the information processing device 1 combines the selected body image with the corrected generated image (by incorporating the corrected generated image into the body image above the neck) to generate a combined body image. The generated combined body image is displayed on the body selection screen.

[0039] A background selection button is displayed on the body selection screen. After the user selects a body image and generates a body combination image, the user can press the background selection button. When the background selection button is pressed, the processing unit 11 of the information processing device 1 displays the background selection screen (see FIG. 17 ) on the display unit 13.

[0040] FIG. 17 is an explanatory diagram showing an example of a background selection screen. The background selection screen displays a background combination image, which is a combination of the body combination image generated on the body selection screen and each of multiple background images (frame images). The processing unit 11 of the information processing device 1, for example, acquires the current date from the clock function unit of the information processing device 1 and displays, as options, background images stored in the background image table 124 that have a display period that includes the current date on the background selection screen. As shown in FIG. 17 , a background image option, "No Background," which displays nothing but the body combination image in the background combination image, may be selectable. The processing unit 11 may also display, as options, background images corresponding to the current time on the background selection screen. The processing unit 11 may also acquire user attributes, for example, by communicating with the user's smartphone, and display background images corresponding to the acquired attributes on the background selection screen. For example, if the current date is within one week before or after the user's birthday, a birthday-themed background image may be displayed on the background selection screen. Furthermore, the processing unit 11 may display, as background images, images acquired, for example, by communicating with the user's smartphone on the background selection screen. Each background combination image is displayed with an associated check box for accepting the selection of the background combination image to be printed.

[0041] A print button is displayed on the background selection screen, and the user can press the print button after selecting a background combination image. When the print button is pressed, the processing unit 11 of the information processing device 1 causes the printing unit 16 to print the background combination image including the generated image generated based on the user image on printing paper, a sticker for an acrylic stand, or the like. That is, in this embodiment, the background combination image is the image to be printed that corresponds to the generated image selected by the user.

[0042] 18 is a flowchart showing an example of a generated image generation process. The processing unit 11 of the information processing device 1 displays a work selection screen on the display unit 13 (S1). The processing unit 11 accepts a work selection via the input unit 14 (S2). The processing unit 11 displays a user image capture screen on the display unit 13 (S3). The processing unit 11 captures a user image via the capture unit 15 (S4). The processing unit 11 displays a character image selection screen on the display unit 13 (S5). The processing unit 11 accepts a character image selection via the input unit 14 (S6).

[0043] The processing unit 11 of the information processing device 1 inputs the user image captured in S4 into the prompt extraction model M1 (S7) and outputs a prompt (S8). The processing unit 11 also inputs the user image into the edge detection module M2 (S9) and outputs edge information (S10). The processing unit 11 reads out an image generation model M3 associated with the character of the character image selected in S6 from the character table 122 (S11). The processing unit 11 inputs the user image, prompt, and edge information into the image generation model M3 (S12) and outputs multiple generated images with different degrees of artistic quality (S13). In this embodiment, the processing unit 11 outputs generated images using the image generation model M3. However, the processing unit 11 may also output generated images by, for example, adjusting noise or applying effects to the user image.

[0044] The processing unit 11 of the information processing device 1 displays a quality selection screen on the display unit 13 (S14). The processing unit 11 accepts the selection of the generated image to be printed through the input unit 14 (S15). The processing unit 11 displays a setting item selection screen on the display unit 13 (S16). The processing unit 11 accepts the selection of options for each setting item through the input unit 14 (S17). The processing unit 11 generates a corrected generated image by correcting the generated image based on the selected options (S18). The processing unit 11 displays the corrected generated image on the setting item selection screen on the display unit 13 (S19). Note that in this embodiment, the processing unit 11 executes the processes of S16 and S17 after executing the process of S15. However, the processing unit 11 may execute the processes of S16 and S17 before executing the process of S8, and the option for the item selected in S17 may be included in the prompt.

[0045] The processing unit 11 of the information processing device 1 reads multiple body images from the body image table 123 (S20) and displays a body selection screen on the display unit 13 (S21). The processing unit 11 accepts the selection of a body image through the input unit 14 (S22). The processing unit 11 generates a body combination image by combining the corrected generated image and the body image based on the selected body image (S23). The processing unit 11 displays the body combination image on the body selection screen on the display unit 13 (S24). The processing unit 11 acquires the current date (S25). The processing unit 11 reads a background image from the background image table 124 that has a display period that includes the current date (S26), and displays the background selection screen on the display unit 13 (S27). The processing unit 11 accepts the selection of a background image through the input unit 14 (S28). The processing unit 11 generates a background combination image by combining the body combination image and the background image based on the selected background image (S29). The processing unit 11 displays the background combination image on the background selection screen on the display unit 13 (S30). The processing unit 11 causes the printing unit 16 to print the background combination image displayed on the background selection screen (S31), and ends the processing.

[0046] According to the above configuration and processing, an image generation model that has learned character images can output a generated image of a user's image that matches the worldview of the work in which the character appears. Furthermore, the user can correct the output generated image to suit their preferences and print the corrected image, allowing it to be used for various purposes. Note that a portion of the processing performed by the information processing device 1 may be executed, for example, on a cloud server 4 (see FIG. 28 ). In this case, the information processing device 1 may transmit a captured user image to the cloud server 4, and the cloud server 4 may generate a generated image by inputting the acquired user image into the image generation model M3. That is, the functions of the information processing device 1 may be realized by multiple devices, including a photographing device, a printing device, a server device, or a computer. Furthermore, the information processing device 1 may be a smartphone, a tablet terminal, a personal computer, or the like. In this case, since the information processing device 1 does not include a printing unit 16, the generated image is transmitted to another printing device, and the generated image is printed. Even when the information processing device 1 is a smartphone, a tablet terminal, a personal computer, or the like, a portion of the processing performed by the information processing device 1 may be executed, for example, on a cloud server 4 (see FIG. 28 ). The information processing device 1 may also be a print sticker machine including Purikura (registered trademark).

[0047] (Embodiment 2) A processing unit 11 of an information processing device 1 according to embodiment 2 accepts a selection of a use of a generated image and creates a generated image according to the selected use. The present invention according to embodiment 2 will be described below with reference to the drawings. Among the components according to embodiment 2, components similar to those of embodiment 1 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0048] 19 is an explanatory diagram showing an example of an image generation model M3 according to the second embodiment. The prompt input to the image generation model M3 according to the second embodiment includes the purpose of the generated image selected by the user (see FIG. 20 ). The purposes included in the prompt include, for example, "business scene," "daily life," or "hobbies." The image generation model M3 generates and outputs one generated image based on the input prompt and edge information.

[0049] FIG. 20 is an explanatory diagram showing an example of a character image selection screen according to the second embodiment. The character image selection screen according to the second embodiment displays character images for multiple characters as well as options for the use of the generated image. Each option is accompanied by a check box that allows the user to select the use of the generated image. When one of the check boxes for the use options is pressed, the start generation button becomes pressable. When the start generation button is pressed, the processing unit 11 of the information processing device 1 reads out an image generation model M3 for the character of the selected character image based on the character table 122, and inputs the captured user image, prompts extracted from the user image, and edge information into the read image generation model M3. Note that the prompt input to the image generation model M3 includes the use of the selected generated image. When a generated image is output by the image generation model M3, the processing unit 11 displays a quality selection screen (see FIG. 14 ) on the display unit. Note that the processing unit 11 of the information processing device 1 may also accept the selection of the use of the generated image by inputting a sentence or word via the input unit 14. At this time, the input sentence or word is included in the prompt input to the image generation model M3.

[0050] In this embodiment, the purpose of the generated image selected by the user includes, but is not limited to, a prompt input to the image generation model M3. The processing unit 11 of the information processing device 1 may create a generated image according to the purpose by, for example, varying the degree of reflection of edge information in the generated image output by the image generation model M3 depending on the purpose of the generated image selected by the user. For example, the processing unit 11 generates a generated image by setting the degree of reflection of edge information to be high when the purpose is "business scene" and low when the purpose is "daily life" or "hobby." This makes the generated image for the "business scene" similar to the user image, thereby increasing the reliability of identity authentication. Furthermore, the processing unit 11 may generate generated images for multiple purposes using the image generation model M3, display them on the display unit 13, and accept a selection of the generated image to be printed from the multiple generated images for the multiple purposes.

[0051] (Embodiment 3) The image generation model M3 according to embodiment 3 is trained using character images of multiple characters appearing in a single work. The processing unit 11 of the information processing device 1 accepts a user's selection of a work, including a manga, anime, movie, or game (see FIG. 11 ), thereby identifying a work to which the user's image's style is to be modeled. The image generation model M3 is fine-tuned (additionally trained) based on the character images (content) included in the identified work, and outputs multiple types of mutually different generated images. That is, the image generation model M3 has learned the style of the work, and when a user's image is input, it outputs a generated image in which the input user's image is modeled in the style of the work. The present invention according to embodiment 3 will be described below with reference to the drawings. Among the components according to embodiment 3, components similar to those of embodiment 1 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0052] FIG. 21 is an explanatory diagram showing an example of a training method for the image generation model M3 according to the third embodiment. In this embodiment, the image generation model M3 is trained to output images that reflect the artistic style of each work by fine-tuning (additional learning) for each work based on the character images (content) of multiple characters included in each work. When the image generation model M3 is configured using stable diffusion, the image generation model M3 is trained (pre-trained) using images obtained from the copyright holder or publicly available images. The processing unit 11 of the information processing device 1 fine-tunes the image generation model M3, which has been pre-trained by scraping, for each work using the character images of multiple characters included in each work as training data, thereby training the image generation model M3 to output images that reflect the artistic style of each work. The image generation model M3 may also be fine-tuned on a server device external to the information processing device 1. Furthermore, the content that may be included in the training data for the image generation model M3 according to this embodiment is not limited to character images related to character images appearing in a work, but may also be an image related to a scene in a work (background image) or an image related to a scene in a work in which a character appears (background image including a character image).

[0053] The image generation model M3 is fine-tuned, for example, by Low-Rank Adaptation (LORA). Specifically, by adding a low-rank decomposition matrix based on character images and captions (character image features) extracted from the character images to each layer of the image generation model M3, the image generation model M3 is fine-tuned (additional learning) to output images that reflect the artistic style of the work. That is, the image generation model M3 is fine-tuned to learn the artistic style of the work based on character images of multiple characters. The image generation model M3 is fine-tuned using a dataset of character images of multiple characters for each work, and multiple image generation models M3 (M3A, M3B, M3C, etc.) that have learned the artistic style of each work are generated. The image generation model M3 may also be trained using, for example, prefix tuning or prompt tuning. Furthermore, the dataset according to this example may include multiple character images of the same character.

[0054] 22 is an explanatory diagram showing an example of a work table 121 according to embodiment 3. The work table according to embodiment 3 includes an image generation model field. The image generation model field stores the type of image generation model M3 (one of M3A, M3B, M3C, etc.) that has been fine-tuned using character images of multiple characters that appear in the corresponding work.

[0055] The processing unit 11 of the information processing device 1 according to the third embodiment causes the photographing unit 15 to photograph a user image, then reads out an image generation model M3 relating to the work selected on the work selection screen based on the work table 121, and inputs the photographed user image, the prompt extracted from the user image, and edge information into the read image generation model M3. That is, after the processing unit 11 causes the photographing unit 15 to photograph a user image, the processing unit 11 outputs an image generated by the image generation model M3 read out from the work table and displays a work level selection screen (see FIG. 14) on the display unit 13 without displaying a character image selection screen (see FIG. 13). Note that the processing unit 11 may input at least one of the user image, the prompt, and the edge information into the image generation model M3.

[0056] FIG. 23 is a flowchart showing an example of a generated image generation process according to the third embodiment. The processes in S41 to S44 are similar to the processes in S1 to S4 shown in FIG. 18. The processing unit 11 inputs the user image captured in S44 into the prompt extraction model M1 (S45) and outputs a prompt (S46). The processing unit 11 also inputs the user image into the edge detection module M2 (S47) and outputs edge information (S48). The processing unit 11 reads out the image generation model M3 associated with the work selected in S42 from the work table 121 (S49). The processing unit 11 inputs the user image, prompt, and edge information into the image generation model M3 (S50) and outputs multiple generated images with different degrees of quality (S51). The processes in S52 to S69 are similar to the processes in S14 to S31 shown in FIG. 18. In this embodiment, processing unit 11 executes the processes of S54 and S55 after executing the process of S53, but processing unit 11 may execute the processes of S54 and S55 before executing the process of S46, and may include options for the item selected in S55 in the prompt. That is, processing unit 11 accepts selection of setting items to be reflected in the generation of a generated image from a plurality of setting items and setting values ​​for each setting item, generates a generated image according to the accepted setting items and setting values, and displays the generated generated image.

[0057] The processing unit 11 of the information processing device 1 may cause the input unit 14 to accept input of a character image selection. At this time, the selected character image may be used as input data for the image generation model M3, and the prompt may include an instruction to move the generated image toward the character image. Furthermore, the image generation model M3 in this embodiment may be trained using training data in which a proportion of character images relating to a specific character is increased among multiple character images. At this time, the processing unit 11 may read out an image generation model M3 with a high proportion of character images of the selected character, and output a generated image using the read image generation model M3.

[0058] (Fourth embodiment) A processing unit 11 of an information processing device 1 according to a fourth embodiment stores a generated image generated by an image generation model M3 in a memory. The present invention according to the fourth embodiment will be described below with reference to the drawings. Among the components according to the fourth embodiment, components similar to those of the first embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

[0059] FIG. 24 is an explanatory diagram illustrating an overview of a generated image utilization system S according to a fourth embodiment. The generated image utilization system S includes, for example, an information processing device 1, a storage medium 2, and a user terminal 3. The information processing device 1 according to the fourth embodiment transmits a generated image to the storage medium 2 and stores it therein. The storage medium 2 is, for example, configured with a Near Field Communication (NFC) tag and is a medium capable of storing image data, incorporated into a key chain or card. In the following description, the storage medium 2 is assumed to be an NFC-embedded card (IC card). The information processing device 1 communicates with the storage medium 2 via the output unit 17 and stores the generated image generated by the processing unit 11 in the storage medium 2. The storage medium 2 is, for example, manufactured by another device and then distributed to a user. The user can bring the storage medium 2 and place it in a predetermined position on the information processing device 1 to receive data related to the generated image from the information processing device 1. Note that the information processing device 1 may store the generated image in the storage medium 2 and then eject the storage medium 2 to provide it to the user. The generated image may be a body combination image or a background combination image that has been processed based on the generated image. In the following description, the generated image will be described as including the generated image, the body combination image, or the background combination image.

[0060] The user terminal 3 is, for example, a smartphone owned by the user. The user terminal 3 may also be a tablet terminal, a personal computer, a wearable device, wearable glasses, or AR goggles. The user terminal 3 is capable of reading the generated image stored in the storage medium 2 from the storage medium 2 via short-range wireless communication.

[0061] 25 is an explanatory diagram showing an example of a generated image transmission screen. After receiving a background selection on the background selection screen, the processing unit 11 of the information processing device 1 according to the fourth embodiment displays the generated image transmission screen on the display unit 13. The generated image transmission screen displays a background combination image (generated image), a user ID input field, an access key input field, and a send button.

[0062] The processing unit 11 accepts input of a user ID from the user in a user ID input field via the input unit 14. The processing unit 11 may acquire the user ID, for example, by communicating with a storage medium 2 that stores the user ID in advance or a user terminal 3. The processing unit 11 also accepts input of an access key (passcode) from the user in an access key input field via the input unit 14. The processing unit 11 may generate a random access key and display the generated access key on a generation screen transmission screen to present it to the user.

[0063] When the send button is pressed on the generated image transmission screen, the processing unit 11 of the information processing device 1 transmits the background combination image (generated image), as well as the input user ID (user information) and access key to the storage medium 2. If the information processing device 1 is a photo ID machine, the storage medium 2 is placed in a predetermined position (NFC reader) within the photo ID machine, and the processing unit 11 of the information processing device 1 transmits the background combination image (generated image), as well as the input user ID and access key to the storage medium 2 placed in the predetermined position. The processing unit 11 may assign a generated image ID to the generated image and transmit the generated image ID to the storage medium 2. The predetermined position where the storage medium 2 is placed is the position where the output unit 17 is located.

[0064] 26 is an explanatory diagram showing an example of a reading screen on the user terminal 3. The user terminal 3 stores the user ID of the user who owns the user terminal 3 in advance. The user terminal 3 accepts input of an access key on the reading screen. The user terminal 3 transmits the user ID stored in advance and the input access key to the storage medium 2. If the user ID and access key stored in the storage medium 2 match the user ID and access key received from the user terminal 3, the storage medium 2 transmits the user ID and generated image stored in the storage medium 2.

[0065] FIG. 27 is an explanatory diagram showing an example of a generated image display screen on the user terminal 3. The user terminal 3 displays the generated image acquired from the storage medium 2. The user terminal 3 also displays multiple function buttons corresponding to the usage of the generated image. The multiple function buttons include, for example, a print button, an output button, and an upload button. When the print button is pressed, the user terminal 3 transmits the generated image to a printer, for example, via communication, and causes the printer to print the generated image. When the output button is pressed, the user terminal 3 outputs the generated image to, for example, another terminal device, a projector (projection device), or a display (display device). When the upload button is pressed, the user terminal 3 outputs (uploads) the generated image to another system via a network (not shown). The other system may include, for example, a web page where multiple users share the generated image, or a metaverse space where avatars based on the generated image appear.

[0066] Fifth Embodiment A processing unit 11 of an information processing device 1 according to a fifth embodiment stores a generated image generated by an image generation model M3 in a memory. The present invention according to the fifth embodiment will be described below with reference to the drawings. Among the components according to the fifth embodiment, components similar to those of the first or fourth embodiment are designated by the same reference numerals, and detailed description thereof will be omitted.

[0067] 28 is an explanatory diagram showing an overview of a generated image utilization system S according to embodiment 5. The generated image utilization system S according to embodiment 5 includes a cloud server 4. The information processing device 1, the user terminal 3, and the cloud server 4 can communicate with each other via a network N. The information processing device 1 communicates with other devices via the network N via a communication unit 18.

[0068] An information processing device 1 according to a fifth embodiment assigns identification information including a user ID and a generated image ID to a generated image, and transmits the generated image and the identification information assigned to the generated image to a cloud server 4. The cloud server 4 stores a generated image table 41 in which the generated image and the identification information corresponding to the generated image are recorded. That is, the cloud server 4 functions as a data server. The information processing device 1 also transmits the identification information to a storage medium 2 for storage. A user terminal 3 reads the identification information from the storage medium 2 and transmits the read identification information to the cloud server 4. The cloud server 4 transmits a generated image corresponding to the received identification information to the user terminal 3. Note that the cloud server 4 may be a device whose functions are realized by multiple server devices, or may be a device corresponding to a node on a blockchain.

[0069] 29 is an explanatory diagram showing an example of a generated image transmission screen according to embodiment 5. The generated image transmission screen according to embodiment 5 displays a background combination image (generated image), a generated image ID, a user ID input field, and a send button. The processing unit 11 of the information processing device 1 assigns the generated image ID to the generated generated image. When the send button is pressed on the generated image transmission screen, the processing unit 11 transmits the generated image and identification information including the input user ID and the assigned generated image ID to the cloud server 4.

[0070] 30 is an explanatory diagram showing an example of a generated image table 41. The generated image table 41 records generated images received by the cloud server 4 from the information processing device 1, and identification information (user ID and generated image ID) assigned to the generated images. The management items (fields) of the generated image table 41 include a user ID field, a generated image ID field, and a generated image field. The user ID field stores the user ID included in the identification information assigned to the generated image. The generated image ID field stores the generated image ID included in the identification information assigned to the generated image. The generated image field stores the generated image received by the cloud server 4 from the information processing device 1, for example, in file format.

[0071] FIG. 31 is an explanatory diagram showing an example of a reading screen on a user terminal 3 according to the fifth embodiment. The user terminal 3 reads the user ID and generated image ID stored in the storage medium 2 from the storage medium 2 via short-range wireless communication. The reading screen according to the fifth embodiment displays the user ID and generated image ID read by the user terminal 3 from the storage medium 2. The reading screen also displays a generated image acquisition button. When the generated image acquisition button is pressed, the user terminal 3 transmits the user ID and generated image ID displayed on the reading screen to the cloud server 4 via the network N. The cloud server 4, having received the user ID and generated image ID, reads the generated image corresponding to the received user ID and generated image ID from the generated image table 41 and transmits the read generated image to the user terminal 3. The user terminal 3, having received the generated image, displays a generated image display screen (see FIG. 27 ).

[0072] In this embodiment, the user ID and generated image ID may not be read from the storage medium 2 by the user terminal 3, but may also be read by the information processing device 1, another photo ID machine for the information processing device 1, or another terminal device for the user terminal 3. The information processing device 1 may also accept an access key input from the user and store the input access key in the storage medium 2 in association with the generated image ID. In this case, the user terminal 3 may be configured to accept an access key input from the user, and transmit the user ID and generated image ID to the cloud server 4 and receive the generated image ID from the cloud server 4 only if the input access key matches the access key read from the storage medium 2.

[0073] (Embodiment 6) An information processing device 1 according to embodiment 6 transmits a generated image to a cloud server 4. The cloud server 4 according to this embodiment is a server that executes processing related to a virtual space platform. The cloud server 4 creates an avatar of the user based on the generated image acquired from the information processing device 1, and displays the created avatar in the virtual space. The present invention according to embodiment 6 will be described below with reference to the drawings. Of the configuration according to embodiment 6, configurations similar to those of embodiment 1 or 5 are designated by the same reference numerals, and detailed description thereof will be omitted.

[0074] FIG. 32 is an explanatory diagram showing an example of an avatar generation model M4. The storage unit 12 (see FIG. 2) of the information processing device 1 according to the sixth embodiment stores the avatar generation model M4. The avatar generation model M4 is a learning model capable of outputting a three-dimensional object when a two-dimensional planar image, such as NeRF (Neural Radiance Fields) or 3D Gaussian Splatting, is input. The processing unit 11 of the information processing device 1 inputs a body combination image (see FIG. 16) generated based on the generated image generated by the image generation model M3 to the avatar generation model M4, and causes the avatar generation model M4 to output a user avatar based on the input generated image.

[0075] FIG. 33 is an explanatory diagram showing an example of an avatar display screen. After accepting a selection of a body image to be combined with a generated image on the body selection screen shown in FIG. 16 , for example, the processing unit 11 of the information processing device 1 displays the avatar display screen on the display unit 13. The avatar display screen displays a body combination image that combines the generated image generated by the image generation model M3 with the body image selected on the body selection screen, and a three-dimensional user avatar output by the avatar generation model M4. The avatar display screen also displays a send button. When the send button is pressed, the processing unit 11 transmits the body combination image and three-dimensional object data of the user's avatar displayed on the avatar display screen to the cloud server 4.

[0076] 34 is an explanatory diagram showing an example of a generated image table 41 according to embodiment 6. Management items of the generated image table 41 according to embodiment 6 include an avatar field. The avatar field stores data of three-dimensional avatar objects received by the cloud server 4 from the information processing device 1, for example, in file format. Note that the generated image field of the generated image table 41 according to this embodiment stores body combination images (generated images) received by the cloud server 4 from the information processing device 1, for example, in file format.

[0077] FIG. 35 is an explanatory diagram showing an example of a virtual space display screen on the user terminal 3. The cloud server 4 transmits to the user terminal 3 the virtual space in which the cloud server 4 executes platform-related processing and the user's avatar stored in the generated image table. The user terminal 3 displays the virtual space on the display unit 13 and displays (appears on) the user's avatar in the virtual space. Image data representing the virtual space and the avatar is generated by a virtual space processing program stored on the cloud server 4 or an application program stored on the user terminal 3. When image data of the virtual space is generated by the virtual space processing program on the cloud server 4, the image data of the virtual space is transmitted to the user terminal 3. Note that the virtual space displayed on the user terminal 3 may display (appear on) the avatar of a user other than the user who owns the user terminal 3.

[0078] An avatar generated by the information processing device 1 may be assigned an NFT (Non-Fungible Token). In this case, the information processing device 1 transmits information related to the NFT assigned to the avatar to the cloud server 4, in addition to the generated avatar. The cloud server 4 stores the avatar and the NFT in a generated image table 41 stored in the cloud server 4, and shares information related to the NFT (the creator of the avatar) with user terminals 3 of multiple users or multiple server devices, thereby managing the information related to the NFT in a distributed manner. Furthermore, when an avatar is displayed (appears) in a virtual space on a user terminal 3 owned by a user other than the user who created the avatar, information related to the NFT assigned to the avatar may be displayed in association with the avatar. Furthermore, when the cloud server 4 receives a request from the user terminal 3 to have the user's avatar appear in the virtual space, for example, the cloud server 4 may perform holder authentication of the NFT assigned to the avatar on the user terminal 3. If the authentication is successful (if the user who owns the user terminal 3 is confirmed to be the NFT holder), the cloud server 4 may display the user's avatar in the virtual space. This allows the cloud server 4 to allow only avatars properly generated by the information processing device 1 (proper avatars generated by the system) to appear in the virtual space, and to prevent unauthorized avatars, such as avatars related to duplicated data or avatars whose creators cannot be identified, from appearing in the virtual space. Note that if a user ID transmitted by a user, for example, via the user terminal 3, matches a user ID stored in the generated image table 41, the cloud server 4 may allow an avatar related to the matching user ID to appear in the virtual space.

[0079] Seventh Embodiment An information processing device 1 according to a seventh embodiment calculates the similarity between a generated image generated by an image generation model M3 and a character image used to train the image generation model M3, and restricts the use of a generated image whose similarity is equal to or greater than a predetermined threshold. The present invention according to the seventh embodiment will be described below with reference to the drawings. Of the components according to the seventh embodiment, components similar to those of the first embodiment will be assigned the same reference numerals, and detailed description thereof will be omitted.

[0080] FIG. 36 is an explanatory diagram showing an example of a similarity calculation model M5. The similarity calculation model M5 is stored in the storage unit 12 (see FIG. 2 ) of the information processing device 1. The similarity calculation model M5 is a learning model capable of extracting image features, such as a CNN or a Vision Transformer. One of the generated images generated by the image generation model M3 and one of the character images used in training the image generation model M3 that output the generated image are input to the similarity calculation model M5. When the similarity calculation model M5 is configured as a neural network including a CNN, the input layer of the similarity calculation model M5 has multiple neurons that accept input of pixel values ​​of the generated image and the character image, and passes the input pixel values ​​to the intermediate layer. The intermediate layer has multiple neurons that extract image features of the generated image and the character image, and passes the extracted image features to the output layer. The output layer calculates and outputs the similarity between the input generated image and the character image based on the image features of the generated image and the character image. The similarity is expressed by a value between 0 and 1, with a higher value indicating that the two images are more similar.

[0081] The processing unit 11 of the information processing device 1 inputs each of the multiple generated images, which have different degrees of reflection of edge information, into the similarity calculation model M5 along with all of the character images used in training the image generation model M3, and calculates the similarity between each of the generated images and the character images. The processing unit 11 may also extract feature points or feature quantities from the images and output the similarity between the generated images and the character images using a processing module that extracts feature points or feature quantities from the multiple images and performs pattern matching based on the feature points or feature quantities extracted from the multiple images. The similarity calculation model M5 may also have a segmentation function that extracts the positions of people (characters or users) in the images as segments, and output the similarity between the segments extracted in the generated images and the character images.

[0082] FIG. 37 is an explanatory diagram showing an example of a quality selection screen according to the seventh embodiment. The processing unit 11 of the information processing device 1 displays, on the quality selection screen, generated images generated by the image generation model M3 whose similarity to all character images used in training the image generation model M3 is less than a predetermined threshold (e.g., 0.5), in the same manner as in the first embodiment. Furthermore, the processing unit 11 displays, as restricted images, generated images generated by the image generation model M3 whose similarity to at least one character image used in training the image generation model M3 is equal to or greater than a predetermined threshold (e.g., 0.5), for example, by shading them. Furthermore, generated images whose similarity to a character image is equal to or greater than the threshold are not provided with a checkbox for accepting user selection, and thus cannot be selected, edited, or printed. In other words, the use of generated images whose similarity to a character image is equal to or greater than the threshold is restricted. Generated images whose similarity to a character image is equal to or greater than the threshold may be displayed in a manner that restricts some of their usage, for example, by disallowing printing but allowing transmission to the user terminal 3. Furthermore, the processing unit 11 may not display generated images whose similarity to the character image is equal to or greater than a threshold value on the artwork level selection screen.

[0083] 38 is an explanatory diagram showing an example of a work quality selection screen according to a modification of embodiment 7. When the number of generated images (e.g., three) whose similarities to all character images used in training the image generation model M3 are less than a predetermined threshold (e.g., 0.5) among the generated images generated by the image generation model M3 is less than a predetermined number, the processing unit 11 of the information processing device 1 according to the modification outputs, by the image generation model M3, a generated image (additional generated image) that reflects the edge information of the user image to a higher degree than generated images whose similarities to the character images are equal to or greater than the threshold.

[0084] In the example shown in FIG. 38 , the generated images with a degree of art work of 0.9 (degree of reflection of edge information: 0.1) and a degree of art work of 0.7 (degree of reflection of edge information: 0.3) have similarities to the character image that are equal to or greater than the threshold, and there are two generated images whose similarities to all character images used in training the image generation model M3 are less than the threshold. Therefore, the processing unit 11 generates, for example, one additional generated image using the image generation model M3. The degree of reflection of the edge information of the user image in the additional generated image is greater than the reflection degree of the generated image with the greatest degree of reflection of edge information among the generated images whose similarities to the character image are equal to or greater than the threshold. In the example shown in FIG. 38 , the processing unit 11 causes the image generation model M3 to generate an additional generated image with a degree of reflection of edge information: 0.4 (degree of art work of 0.6), and displays the generated additional generated image on the art work selection screen. In addition, if the similarity between the additional generated image and at least one of the character images used to train the image generation model M3 is above a threshold, the processing unit 11 may further increase the degree of reflection of edge information and cause the image generation model M3 to generate the additional generated image.

[0085] FIG. 39 is a flowchart showing an example of a generated image generation process according to a modification of the seventh embodiment. The processing unit 11 of the information processing device 1 according to this embodiment executes the following process instead of, for example, S7 to S15 shown in FIG. 18 . The processing unit 11 inputs a captured user image into a prompt extraction model M1 (S71) and outputs a prompt (S72). The processing unit 11 also inputs the user image into an edge detection module M2 (S73) and outputs edge information (S74). The processing unit 11 reads out an image generation model M3 associated with the character of the character image selected in S6 from the character table 122 (S75). The processing unit 11 inputs the user image, prompt, and edge information into the image generation model M3 (S76) and outputs multiple generated images with different degrees of artistic quality (S77).

[0086] The processing unit 11 of the information processing device 1 inputs all combinations of each generated image output in S77 and each character image used in training the image generation model M3 into the similarity calculation model M5 (S78), and outputs the similarity for each combination of generated image and character image (S79). The processing unit 11 identifies, among the generated images output in S77, generated images whose similarity to at least one character image used in training the image generation model M3 is equal to or greater than a predetermined threshold as restricted images (S80). Note that the similarity threshold may differ for each work in which the character associated with the character image appears, or for each character image. The processing unit 11 determines whether the number of generated images that are not restricted images (non-restricted images) among the multiple generated images is equal to or greater than a predetermined number (S81). If the number of non-restricted images is equal to or greater than the predetermined number (S81: YES), the processing unit 11 displays a work level selection screen on the display unit 13, including non-restricted images and restricted images whose use is restricted (S82). The processing unit 11 receives, via the input unit 14, a selection of a generated image to be printed from among the generated images that are non-restricted images (S83), and ends the process (proceeds to S16).

[0087] If the number of non-restricted images is less than the predetermined number (S81: NO), the processing unit 11 of the information processing device 1 increases the degree of reflection of edge information on the generated image output by the image generation model M3 (S84) and inputs the user image, prompt, and edge information to the image generation model M3 (S85). The processing unit 11 outputs the generated image (S86) and returns the process to S78. Note that in S78 after the process is returned, all combinations of the generated images output in S86 and the character images used in training the image generation model M3 are input to the similarity calculation model M5. Also, in S84 and S85, the processing unit 11 may reduce the applicability of the model trained by LoRA and output generated images by the image generation model M3.

[0088] In each of the above-described embodiments, a user image or information based on a user image (prompt or edge information) is input to the image generation model M3, but this is not limiting. An image of a character different from the character associated with the character image used to train the image generation model M3, or information based on the image of the character (prompt or edge information) may also be input to the image generation model M3. Furthermore, a generated image generated by the image generation model M3 based on the input user image, or a generated image generated based on the input character image, or information based on the generated image (prompt or edge information) may be input again to the image generation model M3.

[0089] (Variation) As an example of a generated image generation process according to a variation of the seventh embodiment, an information processing method is provided in which a computer is caused to execute a process in which similarity calculation model M5 generates first content using a generation model that generates content, and determines whether or not to permit use of the first content based on the similarity between the first content and a second content. The first content is a generated image, and the second content is data learned based on a combination of all character images used in training image generation model M3 or a copyrighted work. The generated image may correspond to the first content, and character images and data created from the character images may correspond to the second content. In this case, the processing unit 11 of the information processing device 1 may associate (assign) with (store) the identified unrestricted image (generated image) generation information indicating that use of the unrestricted image is permitted (permission information), a work (original work) featuring a character from the character image used in training the image generation model M3 that generated the unrestricted image, that the unrestricted image is a secondary use image (secondary use work), or that the unrestricted image was generated by the image generation model M3, and later display the generation information together with the unrestricted image (generated image). Furthermore, regardless of whether the generated image is a restricted image or an unrestricted image, if the generated image has been licensed for secondary use by the copyright holder of the original work, the processing unit 11 may associate (assign) with (store) the generated image generation information indicating that secondary use has been licensed (secondary use license information), and later display the generation information together with the generated image. Furthermore, the generated image may be displayed with a recognition image indicating permission information, such as a secondary use license mark or a C (Copyright) mark, or permission information or identification information may be added to image data (properties, etc.) representing the generated image. Furthermore, the similarity threshold may be a threshold for permitting use of the generated image to a character image of a work, which is specified by a person who holds rights to the work, such as the author or copyright holder of the work. In other words, the threshold corresponds to a criterion set as a condition for permission for determining whether or not to permit use of the first content.At this time, the processing unit 11 may read a threshold value corresponding to the conditions of the selected work from a plurality of allowable similarity threshold values ​​corresponding to the conditions of the work, and may restrict use of the generated image (first content) if the similarity is equal to or greater than the read threshold value. At this time, the processing unit 11 may store the generated first content and the similarity of the first content to the second content in association with each other.

[0090] In each of the above-described embodiments, it is desirable that the character images used for training the image generation model M3 are photographed at the same angle or the same field of view. The character images may be manually or automatically cropped. The character images may be obtained, for example, by capturing them from a video of the work.

[0091] In each of the above-described embodiments, the image generation model M3 may be trained using character images of multiple characters appearing in a work, and then additionally trained using a character image of a specific character. Furthermore, the image generation model M3 may be trained using character images of multiple characters, and then additionally trained using a character image of a specific work. Additionally, the image generation model M3 may be trained using images of a work, and then additionally trained using a character image. The order and weights of training of the image generation model M3 may be changeable. The image generation model M3 may be trained to output multiple generated images.

[0092] In each of the above-described embodiments, the image generation model M3 may output a moving image by combining multiple generated images. The image generation model M3 may also output one generated image for one user image. After outputting a generated image, the image generation model M3 may further output multiple generated images (alternative images) with different edge information weights from the generated images already output. After outputting a generated image, the image generation model M3 may also output a specified alternative image that is different from the generated images already output.

[0093] In each of the above-described embodiments, the data (input image) input to the image generation model M3 may include at least one of a user image, a prompt, or edge information. In this case, it is desirable that the input image is a user image. Furthermore, multiple user images may be input to the image generation model M3. In this case, the multiple input user images may each have different face photographing directions, angle of view, or photographed body parts. Note that the multiple user images may be photographed by the same user, angle of view, and at the same time. Furthermore, multiple image generation models may be used simultaneously.

[0094] In each of the above-described embodiments, the processing unit 11 may display a work selection screen instead of a character selection screen. In each of the above-described embodiments, when selecting a work, the processing unit 11 may simply identify a work stored in the work table 121 on the work selection screen and start image generation without selecting a character. In each of the above-described embodiments, when accepting the selection of the purpose of the generated image, the processing unit 11 may include a function for automatic selection without manually selecting the purpose, or may have only the automatic selection function without the manual selection function. In each of the above-described embodiments, the setting item field on the setting item screen displays setting items related to the hair, eye color, and glasses of the generated image. However, the setting item field may also display setting items related to at least one of the hair, eye color, and glasses of the generated image. The setting items may also include options related to facial expressions such as joy, anger, sadness, and happiness, lighting, facial direction, angle of view such as facial magnification or shooting range, background color, background image, gender, age group, or body parts. The setting items may also include options related to backgrounds or clothing, such as a soccer player or a businessman. A function may also be provided that allows the setting items to be automatically or manually modified as needed to match the generated image. The automatic correction function includes, for example, generation by AI. Note that the information processing device 1 may accept input for selecting or correcting setting items after the generated image is displayed to the user. Furthermore, the setting items may be selected from options by a learning model that has been trained to determine the selection of setting items that are suitable for the generated image.

[0095] In each of the above-described embodiments, the background selection screen displays a background image corresponding to the current date. However, a background image corresponding to the current time may also be displayed. The background selection screen may also display a background image corresponding to a season, date, or time specified by the user. Furthermore, the background image selection screen may display a background image corresponding to a situation, such as soccer or business. A background image may also be generated or selected by inputting a generated image and conditions into a learning model or a VLM (Vision Language Model) that is trained to output a background image that matches the generated image and conditions when the generated image and conditions, such as season, date, time, or situation, are input. On the background selection screen, a background image may also be selected by inputting text or selecting a check button.

[0096] In each of the above-described embodiments, the use of the generated image is selected by the user, but is not limited to this. The use of the generated image may be output by a learning model that is trained to output the use of the generated image when a sentence or word input by the user is input.

[0097] A program according to one embodiment of the present disclosure is a program that causes a computer to execute a process of returning an output in response to an input, and causes the computer to execute a process of obtaining a style ID associated with the characteristics of the content, obtaining specific items including numerical values ​​or setting items, and determining the output based on the input, the style ID, and the specific items.

[0098] In a program according to an embodiment of the present disclosure, the style ID is an ID for identifying the characteristics of content specified by the work, author, publisher, or style. That is, the style ID may correspond to a code for identifying an image generation model (see FIG. 5 ) or a work ID (see FIG. 7 ). The specific item is a numerical value or a setting item. In this embodiment, the ID for identifying the image generation model is the style ID, which is stored in a table. In other embodiments, the style ID is not limited to a table in a database, but may also be data in a storage unit. The setting item includes at least one or more pieces of information, such as the AI ​​model to be used or information entered on the input screen. The information entered on the input screen includes one or more pieces of information selected or entered, such as the character, hair color, or eye color (see FIGS. 15 to 17 ). The AI ​​model to be used corresponds to the style ID and is identified based on the style ID. One AI model may correspond to one style ID, or multiple AI models may correspond to one style ID. In this case, the AI ​​model may be identified based on the style ID and other setting items (for example, characters appearing in content whose characteristics are related to the style ID). The style ID and the specific item may contain matching data. In this case, it is possible to store one style ID and a specific item in association with each other, but it is preferable that the style ID and the specific item are stored as independent data. The numerical value includes a threshold that can be expressed, for example, as an Int type or a Float type and can be used within the program.

[0099] In one embodiment of the program of the present disclosure, the input includes a user image taken of a user, the content is identified based on the style ID, and the output includes a generated image generated based on the user image and the content.

[0100] In a program according to an embodiment of the present disclosure, a user image captured of a user is included in the input (input image). Furthermore, content is identified based on a style ID. That is, a user image is input, and a generated image is output based on the style ID. At this time, the processing unit 11 of the information processing device 1 may identify the style ID corresponding to the work selected on the work selection screen, and output the generated image based on the identified style ID.

[0101] In one embodiment of the program of the present disclosure, the input includes a first content, a second content is identified based on the style ID, and the identification item includes a similarity between the second content and the first content.

[0102] In a program according to an embodiment of the present disclosure, the first content included in the input may be, for example, an image, and is different from the content identified by the style ID. The second content corresponds to the content and is identified by the style ID. The image of the first content included in the input may be a generated image output based on a user image and the content. The input data may include at least one of a user image, a prompt, or edge information. The user image is an input image. Multiple user images may be input. When multiple user images are input, they may have different facial orientations, may be images of different parts of the user, or may be images of the same person taken at the same angle and at the same time. In this embodiment, the input image is a generated image and is the first content. In this embodiment, the style ID identifies at least one character image used in training the image generation model M3 or the second content, which is content trained therewith. In this embodiment, the specified item is similarity. Note that in embodiments other than this embodiment, the similarity may be dissimilarity, may be greater than a threshold, may be equal to or less than a threshold, or may be less than a threshold. Furthermore, the threshold value includes two values, and the similarity may be equal to or greater than one of the values, or may be equal to or less than the other value. Note that in embodiments other than this embodiment, an error check may be performed. For example, the error check may include checking whether or not a facial feature is missing. In this case, the identification item includes the error to be identified.

[0103] In each of the above-described embodiments, the image may be a two-dimensional image or a three-dimensional image, and may be a still image or a moving image.

[0104] In each of the above-described embodiments, the number of user images acquired by the information processing device 1 is not limited to one, and may be multiple. Examples of combinations of multiple user images include a combination of multiple images in which the user's face is facing different directions, a combination of images in which the user's facial expressions are different, or a combination of an image centered on the user's face with an image of the upper body or the entire body. By using these combinations of multiple images as input data for the image generation model M3, the information processing device 1 can output a wider variety of generated images and can also generate three-dimensional generated images.

[0105] In each of the above-described embodiments, the background image may be an image other than a generated image, such as an image of a character or scene that appears in the same work or author as the content, an image of a character or scene that is generated to resemble the style of the same work or author as the content, or an image of a real person, object, or place.

[0106] In each of the above-described embodiments, the date is not limited to a calendar date expressed in numbers (e.g., January 1, 2025). The date may be a month only, such as January, a specific date expressed in letters, such as New Year's Day or Christmas, or a specific period of time (e.g., Christmas season or winter).

[0107] In each of the above-described embodiments, the user image, the character image, and the generated image are displayed on the artwork level selection screen, but this is not limiting. The display unit 13 of the information processing device 1 may display either the user image or the character image, as well as multiple types of generated images in order of similarity to the user image.

[0108] In each of the above-described embodiments, the setting items may relate to the selection of additional information including facial expression (e.g., smiling or crying), facial direction, gender, age group, clothing, occupation, location, background, composition (e.g., close-up of face, upper body or whole body), lighting, or brightness.

[0109] In each of the above-described embodiments, the processing unit 11 of the information processing device 1 creates a generated image according to a selected purpose, but this is not limited to this. The processing unit 11 may output a generated image according to the selected purpose from among multiple different types of generated images that have been created.

[0110] In each of the above-described embodiments, the access key may be stored in a storage medium different from the storage medium in which the generated image is stored.

[0111] In each of the above-described embodiments, the processing unit 11 of the information processing device 1 may accept a selection of a work from the user. At this time, the work may be linked to a character image. At this time, the image generation model M3 may be trained for each style of the work. That is, the processing unit 11 may output an image generated by the image generation model M3 corresponding to the selected work. In each of the above-described embodiments, the processing unit 11 of the information processing device 1 may accept a selection of an author of the work from the user. At this time, the image generation model M3 may be trained for each style of the author. That is, the processing unit 11 may output an image generated by the image generation model M3 corresponding to the selected author.

[0112] In each of the above-described embodiments, the content may include a copyrighted work. In this case, the processing unit 11 of the information processing device 1 may determine whether to permit use of the generated image based on the degree of similarity between the generated image and the content. The content may also include a copyrighted work whose use is permitted. In this case, the processing unit 11 of the information processing device 1 may determine whether to permit use of the generated image based on criteria set as permission conditions.

[0113] In each of the above-described embodiments, the printing unit 16 of the information processing device 1 may produce a printed product in which the image to be printed is printed on an item by printing the print generation image (image to be printed) output (created) by the processing unit 11 onto an item such as printing paper or a sticker for an acrylic stand.

[0114] The embodiments disclosed herein are illustrative in all respects and should not be considered limiting. The technical features described in each embodiment may be combined with one another, and the scope of the present invention is intended to include all modifications within the scope of the claims and equivalents thereto. Furthermore, independent and dependent claims described in the claims may be combined with one another in any and all combinations, regardless of the reference format. Furthermore, while the claims use a format in which a claim references two or more other claims (multiple claim format), this is not limiting. Multiple claims (multiple multiple claims) that reference at least one other claim may also be used.

[0115] REFERENCE SIGNS LIST 1 Information processing device 11 Processing unit 12 Storage unit 121 Work table 122 Character table 123 Body image table 124 Background image table M1 Prompt extraction model M2 Edge detection module M3 Image generation model M4 Avatar generation model M5 Similarity calculation model 12a Recording medium 13 Display unit 14 Input unit 15 Photography unit 16 Printing unit 17 Output unit 18 Communication unit 2 Storage medium 3 User terminal 4 Cloud server 41 Generated image table N Network P Program S Generated image utilization system

[0116] (Supplementary Note 1) A program that causes a computer to execute the following processes: acquire a user image captured of a user; acquire target content; and output a plurality of mutually different generated images generated based on the user image and the content. (Supplementary Note 2) The program according to Supplementary Note 1, wherein the content is a character image; and the generated image is output by an image generation model that outputs a generated image using the user image and the character image. (Supplementary Note 3) The program according to Supplementary Note 1 or 2, which stores the image generation model corresponding to each character image, which has been additionally trained using the acquired plurality of character images, accepts selection of the target character image, and outputs the generated image by the image generation model corresponding to the selected character image. (Supplementary Note 4) The program according to any one of Supplements 1 to 3, which outputs the generated image based on edge information of the user image. (Supplementary Note 5) The program according to any one of Supplements 1 to 4, which outputs the plurality of mutually different generated images by inputting the edge information extracted from the user image into an image generation model that outputs a generated image using the user image with different degrees of reflection on the generated image. (Supplementary Note 6) The program according to any one of Supplementary Notes 1 to 5, wherein the content is a character image, and the program accepts a selection of a work in which a character appears, displays the character images relating to a plurality of the characters according to the selected work, and accepts a selection of the target character image from the plurality of displayed character images. (Supplementary Note 7) The program according to any one of Supplementary Notes 1 to 6, wherein the content is a character image, and displays one or both of the user image and the character image, and the plurality of types of generated images in order of similarity to the user image, on the same screen.(Supplementary Note 8) The program according to any one of Supplements 1 to 7, which displays a plurality of setting items related to additional information of the generated image, accepts selection of setting items and setting values ​​for each setting item from the plurality of setting items to be reflected in generating or correcting the generated image, generates or corrects the generated image according to the accepted setting items and setting values, and displays the generated or corrected generated image. (Supplementary Note 9) The program according to any one of Supplements 1 to 3, which generates a body combination image by combining the generated image with an image of the body. (Supplementary Note 10) The program according to any one of Supplements 1 to 9, which accepts selection of a background image, and generates a background combination image by combining the generated image with the selected background image. (Supplementary Note 11) The program according to any one of Supplements 1 to 10, which acquires a date, and generates the background combination image by combining the background image according to the date. (Supplementary Note 12) The program according to any one of Supplements 1 to 11, which accepts selection of at least one of the plurality of types of generated images, and prints an image to be printed corresponding to the accepted generated image. (Supplementary Note 13) The program according to any one of Supplements 1 to 12, which accepts a selection of a purpose of the generated image, and creates the generated image according to the selected purpose, or outputs the generated image according to the purpose selected from the plurality of types of generated images that are different from each other. (Supplementary Note 14) The program according to any one of Supplements 1 to 13, which identifies a work including a manga, anime, movie, or game, and outputs a plurality of types of generated images that are different from each other using an image generation model that has been additionally trained based on the content included in the identified work. (Supplementary Note 15) The program according to any one of Supplements 1 to 14, which stores the generated image in a storage medium. (Supplementary Note 16) The program according to any one of Supplements 1 to 15, which associates user information assigned to the user or an access key for retrieving the generated image from the storage medium with the generated image and stores the associated user information in the storage medium, or stores the access key for retrieving the generated image from the storage medium in a storage medium different from the storage medium in which the generated image is stored.(Supplementary Note 17) The program according to any one of Supplements 1 to 16, which assigns identification information to the generated image, saves the generated image and the identification information on a cloud server, and stores the identification information in a storage medium. (Supplementary Note 18) The program according to any one of Supplements 1 to 17, which creates an avatar of the user based on the generated image, and displays the created avatar in a virtual space. (Supplementary Note 19) The program according to any one of Supplements 1 to 18, which calculates a similarity between the content and the generated image, and displays the generated image for which the calculated similarity is less than a predetermined threshold on a display unit. (Supplementary Note 20) The program according to any one of Supplements 1 to 19, which calculates a similarity between the content and the generated image, and limits usage of the generated image for which the calculated similarity is equal to or greater than a predetermined threshold. (Supplementary Note 21) The program according to any one of Supplements 1 to 20, wherein the content is a character image, and wherein a similarity calculation model that outputs a similarity between the character image and the generated image when the character image and the generated image are input, and outputs the similarity between the character image and the generated image. (Supplementary Note 22) The program according to any one of Supplements 1 to 21, wherein edge information of the user image is input to an image generation model that outputs a generated image using the user image and the character image, and wherein if the number of generated images whose similarity is less than a predetermined threshold is less than a predetermined number among the multiple generated generated images, the generated image that has an increased degree of reflection of the edge information extracted from the user image is generated by the image generation model. (Supplementary Note 23) The program according to any one of Supplements 1 to 22, wherein a selection of the character image, a work in which a character of the character image appears, or an author of the work is accepted, and the generated image is output by the image generation model corresponding to the selected character image, the work, or the author. (Supplementary Note 24) The program according to any one of Supplementary Notes 1 to 23, wherein the content includes a copyrighted work, and whether or not to permit use of the generated image is determined according to a similarity between the generated image and the content.(Supplementary Note 25) The program according to any one of Supplements 1 to 24, wherein the content includes a copyrighted work whose use is permitted, and determines whether or not to allow use of the generated image based on criteria set as conditions for permission. (Supplementary Note 26) An information processing method that causes a computer to execute the following processes: generate first content using a generative model that generates content; and determine whether or not to allow use of the first content depending on the similarity between the first content and a second content. (Supplementary Note 27) The information processing method according to Supplementary Note 26, which restricts use of the first content if the similarity is equal to or greater than a threshold. (Supplementary Note 28) The information processing method according to Supplementary Note 26 or 27, which reads a threshold corresponding to an acquired condition from a plurality of thresholds corresponding to the condition, and restricts use of the first content if the similarity is equal to or greater than the read threshold. (Supplementary Note 29) The information processing method according to any one of Supplements 26 to 28, which displays the first content on a display unit if the similarity is less than a threshold, and restricts output of the first content to the display unit if the similarity is equal to or greater than the threshold. (Supplementary Note 30) The information processing method according to any one of Supplements 26 to 29, wherein the generated first content and the similarity of the first content to the second content are stored in correspondence with each other. (Supplementary Note 31) The information processing method according to any one of Supplements 26 to 30, wherein permission information is assigned to the first content whose similarity is less than a threshold. (Supplementary Note 32) The information processing method according to any one of Supplements 26 to 31, wherein the first content is generated by inputting a prompt to the generative model trained using the second content as training data. (Supplementary Note 33) The information processing method according to any one of Supplements 26 to 32, wherein the generative model generates a plurality of first contents having different degrees of reflection of input data to the generative model, and outputs the first content whose similarity is less than a threshold from among the plurality of generated first contents.(Supplementary Note 34) The information processing method according to any one of Supplements 26 to 33, wherein, when the number of first contents whose similarity is less than a threshold among the generated plurality of first contents is less than a predetermined number, the generative model is caused to generate the first content with an increased degree of reflection of the input data. (Supplementary Note 35) The information processing method according to any one of Supplements 26 to 34, wherein the generative model is trained using training data including a plurality of the second contents, and outputs the first content whose similarity to all of the plurality of second contents included in the training data is less than a threshold. (Supplementary Note 36) The information processing method according to any one of Supplements 26 to 35, wherein the first content is associated with generation information including permission information, an original work, a statement that the first content is a secondary use work, a statement that the first content was generated by a generative model, or secondary use permission information, and the corresponding first content and the generation information are stored. (Supplementary Note 37) The information processing method according to any one of Supplements 26 to 36, wherein generation information including permission information, an original work, a statement that the first content is a secondary use work, a statement that the first content has been generated by a generative model, or secondary use permission information is assigned to the first content, and the first content is displayed together with the assigned generation information. (Supplementary Note 38) A program causing a computer to execute a process of returning an output in response to an input, the program causing a computer to execute a process of: acquiring a style ID associated with a feature of the content; acquiring specific items including numerical values ​​or setting items; and determining the output based on the input, the style ID, and the specific items. (Supplementary Note 39) The program according to Supplementary Note 38, wherein the input includes a user image taken of a user, the content is identified based on the style ID, and the output includes a generated image generated based on the user image and the content. (Supplementary Note 40) The program according to Supplementary Note 38 or 39, wherein the input includes a first content, a second content is identified based on the style ID, and the identification item includes a similarity between the second content and the first content.(Supplementary Note 41) An information processing method that acquires a user image captured of a user, acquires target content, and outputs a plurality of mutually different types of generated images generated based on the user image and the content. (Supplementary Note 42) A program that causes a computer to execute a process of acquiring first content generated by a generative model that generates content, and determining whether or not to permit use of the first content depending on the similarity between the first content and second content, which is a work. (Supplementary Note 43) An information processing method that returns an output in response to an input, the information processing method comprising: acquiring a style ID associated with features of the content, acquiring specific items including numerical values ​​or setting items, and determining the output based on the input, the style ID, and the specific items. (Supplementary Note 44) An information processing device comprising: a processing unit that acquires a user image captured of a user, acquires target content, and outputs a plurality of mutually different types of generated images generated based on the user image and the content. (Supplementary Note 45) An information processing device comprising: a processing unit that generates first content using a generative model that generates content, and determines whether or not to allow use of the first content depending on the similarity between the first content and second content that is a work. (Supplementary Note 46) An information processing device comprising: a processing unit that returns an output in response to an input, wherein the processing unit: acquires a style ID associated with features of the content, acquires specific items including numerical values ​​or setting items, and determines the output based on the input, the style ID, and the specific items. (Supplementary Note 47) A method for producing a printed matter, in which a printed matter having the image to be printed printed on an article is produced by using the program described in Supplementary Note 12.

Claims

1. A program that causes a computer to execute the following process: acquire a user image of a user, acquire target content, and output multiple types of mutually different generated images that are generated based on the user image and the content.

2. The program according to claim 1, wherein the content is a character image, and the generated image is output by an image generation model that outputs a generated image using the user image and the character image.

3. The program according to claim 2, which stores the image generation model corresponding to each character image, which has been additionally trained using the acquired multiple character images, accepts the selection of the target character image, and outputs the generated image using the image generation model corresponding to the selected character image.

4. The program according to any one of claims 1 to 3, wherein the generated image is output based on edge information of the user image.

5. The program according to claim 4, wherein the edge information extracted from the user image is input into an image generation model that outputs generated images using the user image with different degrees of reflection in the generated images, thereby outputting the plurality of types of generated images that are different from each other.

6. A program according to any one of claims 1 to 3, wherein the content is a character image, the program accepts a selection of a work in which a character appears, displays the character images relating to a plurality of the characters corresponding to the selected work, and accepts a selection of the target character image from the plurality of displayed character images.

7. A program according to any one of claims 1 to 3, wherein the content is a character image, and the user image and / or the character image, as well as the plurality of types of generated images in order of similarity to the user image, are displayed on the same screen.

8. A program according to any one of claims 1 to 3, which displays a plurality of setting items related to additional information of the generated image, accepts selection of setting items and setting values ​​for each setting item from the plurality of setting items to be reflected in the generation or correction of the generated image, generates or corrects the generated image according to the accepted setting items and setting values, and displays the generated or corrected generated image.

9. A program according to any one of claims 1 to 3, wherein the generated image is a face image, and a body combination image is generated by combining the generated image with a body image.

10. A program according to any one of claims 1 to 3, which accepts a selection of a background image, and generates a background combined image by combining the generated image with the selected background image.

11. The program according to claim 10, which acquires a date and generates the background combination image by combining the background images according to the date.

12. The program according to any one of claims 1 to 3, which accepts the selection of at least one of the plurality of types of generated images, and prints the image to be printed that corresponds to the accepted generated image.

13. A program according to any one of claims 1 to 3, which accepts a selection of the intended use of the generated image, and creates the generated image according to the selected intended use, or outputs the generated image according to the intended use selected from among the plurality of mutually different types of generated images.

14. A program according to any one of claims 1 to 3, which identifies a work including manga, anime, movies, or games, and outputs multiple types of mutually different generated images using an image generation model that has been additionally trained based on the content contained in the identified work.

15. The program according to any one of claims 1 to 3, wherein the generated image is stored in a storage medium.

16. The program according to claim 15, wherein user information assigned to the user or an access key for retrieving the generated image from the storage medium is linked to the generated image and stored in the storage medium, or the access key for retrieving the generated image from the storage medium is stored in a storage medium different from the storage medium in which the generated image is stored.

17. A program according to any one of claims 1 to 3, which assigns identification information to the generated image, stores the generated image and the identification information on a cloud server, and stores the identification information on a storage medium.

18. A program according to any one of claims 1 to 3, which creates an avatar of the user based on the generated image, and displays the created avatar in a virtual space.

19. A program according to any one of claims 1 to 3, which calculates the similarity between the content and the generated image, and displays the generated image on a display unit when the calculated similarity is less than a predetermined threshold.

20. A program according to any one of claims 1 to 3, which calculates the similarity between the content and the generated image, and restricts the use of the generated image when the calculated similarity is equal to or greater than a predetermined threshold.

21. A program according to any one of claims 1 to 3, wherein the content is a character image, and the character image and the generated image are input into a similarity calculation model that outputs the similarity between the character image and the generated image when the character image and the generated image are input, and the similarity between the character image and the generated image is output.

22. The program according to claim 21, wherein edge information of the user image is input to an image generation model that outputs a generated image using the user image and the character image, thereby outputting the generated image; and if the number of generated images having a similarity below a predetermined threshold among the plurality of generated images is less than a predetermined number, the generated image that has a higher degree of reflection of the edge information extracted from the user image is generated by the image generation model.

23. The program according to claim 2, which accepts selection of the character image, a work in which the character of the character image appears, or the author of the work, and outputs the generated image using the image generation model corresponding to the selected character image, work, or author.

24. A program according to any one of claims 1 to 3, wherein the content includes a copyrighted work, and whether or not to permit use of the generated image is determined depending on the degree of similarity between the generated image and the content.

25. The program according to claim 24, wherein the content includes a copyrighted work whose use is permitted, and whether or not to permit use of the generated image is determined based on criteria set as conditions of permission.

26. An information processing method that causes a computer to execute the following process: generate a first content using a generative model that generates content; and determine whether or not to permit use of the first content based on the similarity between the first content and a second content.

27. The information processing method according to claim 26, further comprising restricting use of the first content if the similarity is equal to or greater than a threshold value.

28. An information processing method according to claim 27, further comprising: reading a threshold value corresponding to the acquired condition from a plurality of threshold values ​​corresponding to the condition; and restricting the use of the first content if the similarity is equal to or greater than the read threshold value.

29. An information processing method as described in claim 26 or 27, wherein if the similarity is less than a threshold, the first content is displayed on a display unit, and if the similarity is equal to or greater than the threshold, output of the first content to the display unit is restricted.

30. The information processing method according to claim 26 or 27, wherein the generated first content and the degree of similarity of the first content to the second content are stored in association with each other.

31. The information processing method according to claim 26 or 27, wherein permission information is assigned to the first content whose similarity is less than a threshold value.

32. An information processing method according to claim 26 or 27, wherein the first content is generated by inputting a prompt to the generative model that has been trained using the second content as training data.

33. An information processing method as described in claim 26 or 27, wherein a plurality of first contents are generated using the generative model, each having a different degree of reflection of input data to the generative model, and among the plurality of generated first contents, a first content whose similarity is less than a threshold is output.

34. An information processing method as described in claim 33, wherein, if the number of first contents whose similarity is less than a threshold among the multiple generated first contents is less than a predetermined number, the generative model is caused to generate first contents that have a higher degree of reflection of the input data.

35. The information processing method described in claim 32, wherein the generative model is trained using training data including a plurality of the second contents, and outputs the first content whose similarity to all of the plurality of second contents included in the training data is less than a threshold.

36. An information processing method according to claim 26 or 27, wherein the first content is associated with generation information including permission information, an original work, information indicating that the first content is a secondary use work, information indicating that the first content is generated by a generative model, or secondary use license information, and the corresponding first content and generation information are stored.

37. An information processing method according to claim 26 or 27, further comprising: assigning to the first content generation information including permission information, an original work, a statement that the first content is a secondary use work, a statement that the first content is generated by a generative model, or secondary use permission information; and displaying the first content together with the assigned generation information.

38. A program that causes a computer to execute a process that returns an output in response to an input, the program performing the following process: obtain a style ID associated with the characteristics of the content; obtain specific items including numerical values ​​or setting items; and determine the output based on the input, the style ID, and the specific items.

39. The program of claim 38, wherein the input includes a user image taken of a user, the content is identified based on the style ID, and the output includes a generated image generated based on the user image and the content.

40. The program of claim 38, wherein the input includes a first content, a second content is identified based on the style ID, and the identification item includes a similarity between the second content and the first content.

41. An information processing method comprising: acquiring a user image of a user; acquiring target content; and outputting a plurality of mutually different generated images generated based on the user image and the content.

42. A program that causes a computer to execute a process of obtaining first content generated by a generative model that generates content, and determining whether or not to permit use of the first content based on the degree of similarity between the first content and second content, which is a copyrighted work.

43. An information processing method that returns an output in response to an input, comprising: acquiring a style ID associated with the characteristics of the content; acquiring specific items including numerical values ​​or setting items; and determining the output based on the input, the style ID, and the specific items.

44. An information processing device comprising a processing unit that acquires a user image captured of a user, acquires target content, and outputs multiple types of generated images that are different from each other and are generated based on the user image and the content.

45. An information processing device comprising: a processing unit that generates first content using a generative model for generating content; and determines whether or not to permit use of the first content based on the degree of similarity between the first content and second content, which is a copyrighted work.

46. ​​An information processing device having a processing unit that returns an output in response to an input, wherein the processing unit: acquires a style ID associated with the characteristics of content; acquires specific items including numerical values ​​or setting items; and determines the output based on the input, the style ID, and the specific items.

47. A method for producing a printed matter, which uses the program according to claim 12 to produce a printed matter on an article on which the image to be printed is printed.

Images