Image generation device, image generation method, and image generation program

Abstract

It allows for a simple and accurate understanding of how the subject is combined with various items. [Solution] An image generation device 10 that generates a composite image in which a subject included in a subject image is combined with a predetermined item, comprising: an image receiving unit 12 that receives a subject image; an item receiving unit 13 that receives the setting of an item image of an item to be composited with the subject image; and an AI control unit 15 that transmits a composite image generation instruction to an artificial intelligence unit 20 to generate a composite image based on the subject image, the item image and the item type of the item, wherein the AI ​​control unit instructs the artificial intelligence unit to determine the relative position of the item with respect to the subject in the composite image, taking into consideration the item type.

Description

Technical Field

[0001] The present invention relates to an image generation device, an image generation method, and an image generation program.

Background Art

[0002] When deciding on wedding dresses or the like, in addition to the clothing, it is necessary to decide on various items to be worn such as necklaces and earrings, and it is cumbersome to repeat the fitting for each. Therefore, there is a need for a technology that can easily grasp how a subject wears various items.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Patent Document 1 discloses an apparel image generation system including a display unit that displays a selection screen allowing a user to select at least one of a model image, an item image, a background image, a face image, and a pose image, and an image generation unit that generates an apparel image from the images selected by the user using an image generation model.

[0005] An object of the present invention is to provide an image generation device that can easily and accurately grasp how a subject is combined with various items.

Means for Solving the Problems

[0006] To achieve the above objective, an image generation apparatus according to one aspect of the present invention is an image generation apparatus that generates a composite image in which a subject included in a subject image is combined with a predetermined item, comprising: an image receiving unit that receives the subject image; an item receiving unit that receives the setting of an item image of the item to be composited with the subject image; and an AI control unit that transmits a composite image generation instruction to an artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, wherein the AI ​​control unit instructs the artificial intelligence unit to determine the relative position of the item with respect to the subject in the composite image, taking into consideration the item type.

[0007] The subject is a person, the item receiving unit is capable of receiving the item image of the item type being a hairstyle, and the AI ​​control unit, upon receiving the item image of the item type being a hairstyle, may transmit a hairstyle synthesis instruction to the artificial intelligence unit to generate a composite image in which the hairstyle of the subject included in the subject image becomes the hairstyle included in the item image.

[0008] The aforementioned items are stored in a storage unit in advance, linked to loan reservation information. The system further includes an item extraction unit that refers to the loan reservation information of the aforementioned items and extracts the aforementioned items that are available for loan on the desired loan date, and a display control unit that displays only the aforementioned items extracted by the item extraction unit in an item selection field for selection. The item reception unit may accept the setting of the aforementioned items by the selection in the item selection field.

[0009] The aforementioned items are stored in a storage unit in advance in association with inventory information. The system further includes an item extraction unit that refers to the inventory information of the items and extracts items that are in stock, and a display control unit that displays only the items extracted by the item extraction unit in an item selection field so that they can be selected. The item receiving unit may accept the setting of the items by selection in the item selection field.

[0010] The aforementioned items are stored in a memory unit in advance, linked to an index of recommendation level for which selection is recommended. The AI ​​control unit sends a recommendation item extraction instruction to the artificial intelligence unit to output a recommended item for selection, and the display control unit displays the items extracted by the recommendation item extraction instruction in the item selection field.

[0011] The AI ​​control unit may send a related item extraction instruction to the artificial intelligence unit to output related items related to the item selected by the item reception unit, based on the item and item type selected by the item reception unit, and the display control unit may display the extracted related items in the item selection field.

[0012] To achieve the above objective, an image generation method according to another aspect of the present invention is an image generation method for generating a composite image in which a subject included in a subject image is combined with a predetermined item, wherein a computer performs an image reception step of receiving the subject image, an item reception step of receiving the setting of an item image of the item to be synthesized with the subject image, and an AI control step of transmitting a composite image generation instruction to an artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, wherein in the AI ​​control step, the AI ​​control unit is instructed to determine the relative position of the item with respect to the subject in the composite image, taking into consideration the item type.

[0013] To achieve the above objective, an image generation program according to yet another aspect of the present invention is an image generation program that generates a composite image in which a subject included in a subject image is combined with a predetermined item, wherein the program causes a computer to execute an image reception step of receiving the subject image, an item reception step of receiving the setting of an item image of the item to be composited with the subject image, and an AI control step of transmitting a composite image generation instruction to an artificial intelligence unit to generate the composite image based on the subject image, the item image and the item type of the item, wherein the AI ​​control step instructs the artificial intelligence unit to determine the relative position of the item with respect to the subject in the composite image, taking into consideration the item type in the composite image generation instruction. Computer programs can be provided by storing them on various data-readable storage media, or by making them available for download via networks such as the Internet. [Effects of the Invention]

[0014] According to the present invention, it is possible to easily and accurately grasp how a subject is combined with various items. [Brief explanation of the drawing]

[0015] [Figure 1] This figure shows the overall configuration and functional configuration of an image generation device according to an embodiment of the present invention. [Figure 2] This figure shows an example of a fitting screen displayed on a user terminal connected to the image generation device described above. [Figure 3] This figure shows an example of a fitting screen displayed on the user's terminal. [Figure 4] This figure shows an example of a fitting screen displayed on the user's terminal. [Figure 5] This figure shows an example of a fitting screen displayed on the user's terminal. [Figure 6] This is a sequence diagram showing the processing flow performed by the image generation device described above.

Best Mode for Carrying Out the Invention

[0016] Hereinafter, embodiments of an image generation apparatus according to the present invention will be described with reference to the drawings.

[0017] ●Image Generation System● An image generation system is a system that generates a composite image in which a subject included in a subject image is combined with a predetermined item, for example, a composite image in which the subject is wearing an item.

[0018] As shown in FIG. 1, an image generation system 1 includes an image generation apparatus 10, an artificial intelligence unit 20, a user terminal 30 used by a user, and an information storage device 40 that stores information about items to be combined, etc., which are configured to be communicable via a network NW. The image generation apparatus 10 may be constituted by a hardware device, or some or all of its functions may be realized by a cloud computer. Also, each configuration of the image generation apparatus 10 may be realized by an API (Application Programming Interface). In this embodiment, the mutual communication among the image generation apparatus 10, the artificial intelligence unit 20, the user terminal 30, and the information storage device 40 is wireless, but some or all of the connections may be wired. Furthermore, the image generation apparatus 10 may be constituted by a plurality of hardware configurations. In this case, the plurality of hardware configurations may be connected by wire or wirelessly, and information may be transmitted and received between them.

[0019] Also, the artificial intelligence unit 20 has the function of AI (Artificial Intelligence). The image generation apparatus 10 transmits appropriate instructions to the artificial intelligence unit 20 and acquires the information output from the artificial intelligence unit 20.

[0020] ●User Terminal 30 The user terminal 30 is, for example, a smartphone, tablet, or personal computer. The user operating the user terminal 30 may be, for example, a company that rents out wedding attire, a wedding planner who acts as an intermediary between the rental company and the bride and groom, or the customer themselves. Furthermore, the user may be an item vendor or a beauty salon. The user terminal 30 primarily consists of a display unit 31, an operation unit 32, and a communication processing unit 33, comprising a CPU (Central Processing Unit), computer programs executed by the CPU, and RAM (Random Access Memory) and ROM (Read Only Memory) for storing computer programs and predetermined data.

[0021] The display unit 31 is implemented by a display or the like for outputting data. The display unit 31 can appropriately display information from the display control unit 11 of the image generation device 10, for example, using a web browser or an appropriate application. The screen displayed on the display unit 31 will be described later.

[0022] The operation unit 32 is a functional unit for receiving input such as uploading subject images and selecting items, and may be, for example, a keyboard or touch panel display that accepts text input.

[0023] The communication processing unit 33 is a processing unit that enables the image generation device 10 to send and receive data in accordance with a predetermined protocol via a network NW such as the Internet, and is implemented by an application or a web browser.

[0024] ●Information storage device 40 The information storage device 40 is a device that stores items to be composited onto the subject image, customer information, etc. The information storage device 40 includes, for example, a customer database DB41, an item database DB42, and a reservation database DB43. Note that each database DB41 to DB43 may be provided in different devices, and some or all of the information stored in the information storage device 40 may be held by the image generation device 10.

[0025] The customer database DB41 is a database that stores information about customers, and may store customer identification information and other appropriate information such as the customer's address and gender. The customer database DB41 may also store images in which the customer is the subject, linked to the customer's identification information. These images are referenced as subject images in the generation of composite images.

[0026] The item database DB42 is a database that stores information on items that can be provided by companies using the image generation system 1. The types of items include clothing such as dresses, as well as accessories such as necklaces, tiaras, earrings, bracelets, veils, and bouquets. Items may also include hairstyles. The item database DB42 stores item identification information, item images, item types, and information such as the profit margin and recommendation level (as appropriately set by companies using the image generation system 1), all linked together. The item database DB42 may also store item dimensions, or other appropriate information that can identify the item dimensions.

[0027] The reservation database DB43 is a database that stores information regarding item rental reservations, and stores item identification information linked to the rental reservation schedule for that item. In addition, the reservation database DB43 may also link the identification information of the customer for whom the rental is scheduled with the item that has been reserved. Furthermore, the reservation database DB43 may be a database that stores various contract histories related to the customer's wedding, and may store, for example, customer identification information linked to appropriate reservation information other than items, including the location and date and time of the wedding venue where the customer will hold their wedding.

[0028] ●Image generation device 10 The image generation device 10 is an information processing device that generates a composite image in which a subject included in a subject image is combined with a predetermined item, in response to an operation of the user terminal 30. If the subject is a living thing such as a person or a pet, and the item is clothing or accessories, the image generation device 10 generates a composite image in which the subject is wearing them. The image generation device 10 sends instructions to the artificial intelligence unit 20 and obtains output from the artificial intelligence unit 20.

[0029] As shown in Figure 1, the image generation device 10 consists mainly of a display control unit 11, an image receiving unit 12, an item extraction unit 13, an item receiving unit 14, an AI control unit 15, and a communication processing unit 16, comprising a CPU (Central Processing Unit), a computer program executed by the CPU, and RAM (Random Access Memory) and ROM (Read Only Memory) for storing the computer program and predetermined data.

[0030] ●Display control unit 11 The display control unit 11 performs processing to display an appropriate screen related to the image generation system 1 on the user terminal 30. For example, the display control unit 11 performs processing such as generating and sending an HTML (Hyper Text Markup Language) file to display a web page on the user terminal 30. The display control unit 11 may also perform processing such as generating and sending display data for an application that uses the image generation system 1.

[0031] ●Image reception unit 12 The image reception unit 12 is a functional unit that receives subject images. Subject images may be received by uploading them, for example via the user terminal 30, or by receiving them via URL. Subject images obtained by uploading or via URL may be linked to customer identification information and stored in the customer database DB41. This makes it easy to retrieve similar subject images when attempting to generate another composite image later. Furthermore, the image receiving unit 12 may, after receiving the identification information of the customer to be targeted for image generation, refer to the customer database DB41 to obtain subject images associated with the customer's identification information. The image receiving unit 12 may receive multiple subject images. The multiple subject images may be images of the same subject taken from different directions, for example, images taken from the front, at an angle, and from behind. In this case, images taken from multiple directions can be generated simultaneously.

[0032] Furthermore, the image receiving unit 12 may acquire user information from the user terminal 30. The user information may include information linking the identification information of the user, i.e., the lending business operator, with the identification information of the customer who has a contract with the business operator, and the subject image to be used may be identified by selecting the contract customer. If the user is performing a login process on the user terminal 30, the user information may also be extracted from the login information entered during the login process.

[0033] ● Item extraction unit 13 The item extraction unit 13 is a functional unit that extracts candidate items to be displayed on the user terminal 30 from the item database DB 42. For example, the item extraction unit 13 refers to the item rental reservation information and extracts items that can be rented on the desired rental date. The display control unit 11 displays only the items extracted by the item extraction unit 13 in the item selection field. The desired rental date is, for example, the planned wedding date in the case of renting wedding attire. Since the planned wedding date is usually decided before trying on the attire, this configuration makes it possible to select items in a realistic way that is in line with the wedding schedule.

[0034] ● Item reception area 14 The item receiving unit 14 is a functional unit that receives information about items to be composited onto a subject image via the user terminal 30. The item receiving unit 14 may accept uploads via the user terminal 30, or it may accept the URL of the subject image. The item receiving unit 14 may also accept the selection of items to be composited from a list of items displayed on the screen of the user terminal 30. For example, the item receiving unit 14 accepts item identification information along with the item type. The item receiving unit 14 may accept the selection of an item type via the user terminal 30, or, if it accepts the selection of an item stored in the item database DB42, it may obtain information about the item type that has been pre-associated with the item in the item database DB42.

[0035] ● AI control unit 15 The AI ​​control unit 15 is a functional unit that controls the input to the artificial intelligence unit 20. The various instructions transmitted by the AI ​​control unit 15 are, for example, prompts, but are not limited to prompts, as long as they are instructions that the artificial intelligence unit 20 can interpret.

[0036] The AI ​​control unit 15, for example, transmits a composite image generation instruction to the artificial intelligence unit 20 via the communication processing unit 16, which instructs the artificial intelligence unit 20 to generate a composite image based on the subject image, the item image, and the item type of the item. In addition, the AI ​​control unit instructs the artificial intelligence unit 20 to determine the relative position of the item to the subject in the composite image, taking the item type into consideration, in the composite image generation instruction. Since the position in which an item is worn is determined based on the item type, this configuration allows for the generation of a composite image with greater accuracy. That is, for example, distinguishing between a necklace and a bracelet is difficult with only the item image, but with this configuration, it is possible to composite the image to the desired mounting position. The AI ​​control unit 15 may also transmit information regarding the dimensions of the item. This allows for a more accurate representation of the ratio of dimensions between the item and the subject in the composite image. In this case, transmitting the subject's height, etc., along with the item's dimensions, can generate an even more accurate composite image.

[0037] Furthermore, the AI ​​control unit 15 may send an instruction to generate a composite image by combining the costume with a background image that will be used while wearing the costume, such as the background of a wedding venue. The background image may be uploaded, specified by URL, or selected via the user terminal 30, or it may be obtained from the venue identification information included in the reservation information by referring to an appropriate database that stores venue identification information and background images linked in advance. In this case, the background can be considered another example of an item type within the scope of the claims. With such a configuration, it is possible to grasp more accurately how the costume will actually look when worn at the venue.

[0038] Furthermore, if only a background image showing a suitable bride and groom is provided without any specification of attire, the AI ​​control unit 15 may instruct the artificial intelligence unit 20 to generate a composite image in which the subject's face is that of either the bride or groom. In this case, if the AI ​​control unit 15 receives an image showing both the bride and groom as the subject image, it may also instruct it to generate a composite image in which the faces of each of them are those of the bride and groom shown in the background image. With such a configuration, customers can easily get a feel for what a wedding will be like. In turn, it can encourage customers to make a reservation before signing one, and it can also help them in selecting a wedding venue.

[0039] Furthermore, if the AI ​​control unit 15 receives an item image of the item type "hairstyle," it may send a hairstyle synthesis instruction to the artificial intelligence unit 20 to generate a composite image in which the hairstyle of the subject included in the subject image becomes the hairstyle included in the item image. This prevents face synthesis and ensures that only the hairstyle is reflected, even if the item image of the hairstyle includes the face of another person.

[0040] The AI ​​control unit 15 receives the generated composite image from the artificial intelligence unit 20. The composite image may be a front view image, multiple images from different angles, or a video. The composite image may be generated from one subject image, multiple images may be generated from one subject image, or multiple subject images may be read and each image may be generated. The generated composite image is stored in the customer database DB41, for example, linked to the customer's identification information. With this configuration, when the customer tries to generate another composite image, this image can be referenced, and recommended items that take the customer's preferences into account can be extracted.

[0041] Furthermore, while the composite image is displayed, the AI ​​control unit 15 may accept color specifications for the costume via touch input or voice input to the user terminal 30. In this case, the AI ​​control unit 15 instructs the artificial intelligence unit 20 to change only the color of the composite image and obtains the newly generated composite image. With this configuration, even if a store does not have the costume in a different color in stock, it is possible to easily check how the costume looks when tried on without having to order it from the manufacturer or other stores. In addition, the system may determine whether the specified color is in stock at other stores or in the lineup of the costume in question, and if not, it may synthesize and output a costume of a similar color, or it may output a notification that the color is not available.

[0042] Furthermore, the AI ​​control unit 15 sends a recommended item extraction instruction to the artificial intelligence unit 20, which outputs recommended items that are recommended for selection. The AI ​​control unit 15 extracts recommended items from the items available for loan extracted by the item extraction unit 13. The recommendation level can be set using appropriate indicators, such as the degree to which stores want to promote lending or the profit margin, which are stored in the item database DB 42 and linked to the items. It is also possible to calculate the recommendation level by combining multiple indicators. For example, the AI ​​control unit 15 may instruct the artificial intelligence unit 20 to refer to the item database DB 42, determine the recommendation order of products according to the degree to which stores want to promote lending, and then extract the recommended items along with the recommendation order. Alternatively, the AI ​​control unit 15 may instruct the artificial intelligence unit 20 to refer to the item database DB 42 and prioritize the extraction of items with high profit margins. The AI ​​control unit 15 may output recommended items by considering multiple indicators in combination.

[0043] Furthermore, the AI ​​control unit 15 may send a related item extraction instruction to the artificial intelligence unit 20, which outputs related items related to the item selected by the user terminal 30, based on the selected item and item type. Related items are items of a different item type than the previously selected item. Note that the ability to wear multiple items may be set for each item type, and for item types that can be worn multiple times, the system may be configured to allow the extraction of items of the same item type as the previously selected item. This configuration would enable suggestions such as layering necklaces. In addition to extracting related items in the same way as the instructions for recommended items, the AI ​​control unit 15 may extract items that are compatible with the previously selected item as related items through image analysis, or it may extract items that are likely to be rented together after referring to the rental history. With such a configuration, items that match the item selected by the user can be recommended with high accuracy. Recommended items and related items may be displayed in a way that allows them to be distinguished on the try-on screen G10.

[0044] The AI ​​control unit 15 outputs the extracted items along with their recommended order to the artificial intelligence unit 20. The display control unit 11 displays the extracted items on the user terminal 30 in the recommended order.

[0045] ● Artificial Intelligence Department 20 The Artificial Intelligence Unit 20 is an artificial intelligence (AI) equipped with an appropriate learning model. A learning model (also called a machine learning model) refers to a learning model based on a machine learning algorithm. Specific machine learning algorithms include the nearest neighbor method, Naive Bayes method, decision trees, and support vector machines. Another example is deep learning, which uses neural networks to generate features and connection weights for learning. The Artificial Intelligence Unit 20 can apply the above algorithms as appropriate.

[0046] The artificial intelligence unit 20 has a pre-trained model that has been appropriately machine-learned. The training data may be provided by administrators or others, or it may include information collected from the internet or other sources. The pre-trained model has obtained appropriate training data in advance and can also undergo additional training as needed.

[0047] The artificial intelligence unit 20 has, for example, a first trained model that takes a subject image, an item image, and an item type as input and outputs a composite image. Based on a composite image generation instruction from the AI ​​control unit 15, the artificial intelligence unit 20 generates and outputs a composite image based on the subject image and item image using the first trained model.

[0048] The artificial intelligence unit 20 may have a second trained model that takes an item and item type as input and outputs related items associated with that item. In this case, based on the related item extraction instruction from the AI ​​control unit 15, the artificial intelligence unit 20 outputs related items based on the item selected at the user terminal 30 and its item type using the second trained model.

[0049] The communication processing unit 16 is a processing unit that enables the transmission and reception of data with the user terminal 30 via a network NW such as the Internet, in accordance with a predetermined protocol. For example, the communication processing unit 16 receives subject images from the user terminal 30. The communication processing unit 16 also transmits the item images of clothing and accessories selected by the user terminal 30 to the image generation device 10, linking them to their item types. The communication processing unit 16 also receives a composite image and transmits the received composite image to the user terminal 30.

[0050] ●Screen example Figures 2 to 5 show examples of the fitting screen G10 displayed on the user terminal 30 by the display control unit 11. Figures 2 and 3 are displayed vertically in this order on a web page, and the displayed area can be continuously moved by appropriate scrolling operations.

[0051] As shown in Figure 2 or Figure 3, the try-on screen G10 displays the subject image setting field G11, the costume setting field G12, the accessory setting field G13, the costume type setting field G14, the generated image display field G15, and the generation start button G16, etc. Note that the costume setting field G12 and the accessory setting field G13 are examples of item selection fields in the claims, respectively.

[0052] The subject image setting field G11 is an area where the subject image is set. The subject image can be accepted, for example, by uploading a file or specifying a URL. Alternatively, as mentioned above, the subject image associated with a customer may be selected by specifying the customer's identification information.

[0053] The costume settings section G12 is the area for accepting costume settings. The accessory settings section G13 is the area for accepting settings for items other than costumes, i.e., accessories. Costumes and accessories may be accepted by uploading files or specifying URLs, or they may be selectable from a list of costumes available for rental for logged-in users. Alternatively, only costumes available for rental on the customer's requested rental date may be displayed for selection.

[0054] In the accessory settings section G13, the type of accessory to be set is displayed for selection. In the diagram, one of the following can be selected: necklace, tiara, earrings, bracelet, veil, or bouquet. In the accessory settings section G13, for example, one accessory can be set for each accessory type, and multiple accessories of different types can be set for a single subject. Note that accessories of a type that can be worn multiple times may accept multiple settings. Also, when an accessory type is selected, accessories of the selected type from the items available for loan in the item database DB42 may be displayed. In the costume settings section G12 and the accessory settings section G13, the display control unit 11 may display costumes and accessories according to the recommended order output from the artificial intelligence unit 20.

[0055] Furthermore, in the accessory setting field G13, when the display control unit 11 receives the setting of an item in the item reception unit 14, related items of the selected item may be displayed. Related items are output from the artificial intelligence unit 20, for example, in response to a related item extraction instruction from the AI ​​control unit 15. There may be one or more related items, and they may be displayed in descending order of their degree of relevance. In addition, items may be pre-associated with related items in the item database DB42, and displayed based on this information.

[0056] The costume type setting area G14 is an area where the selection of the costume type for the set costume is received. In this figure, the costume type can be selected from upper body, lower body, full body, or shoes. The costume type is another example of an item type. The generated image display area G15 is an area where the composite image obtained from the artificial intelligence unit 20 is displayed. The generation start button G16 is an operator that starts the generation of a composite image in which the subject in the subject image is wearing the set costume and accessories. In the example shown in this figure, the composite image is generated when the generation start button G16 is selected, but instead, image generation may be performed each time an item is selected and displayed in the generated image display area G15.

[0057] Figure 4 shows the fitting screen G10 after the subject image and costume settings have been accepted in the subject image setting area G11 and costume setting area G12 of Figure 2. Figure 5 shows the fitting screen G10 that is displayed when the start generation button G16 is selected after the accessory and costume type settings have been accepted in the accessory setting area G13 and costume type setting area G14 of Figure 3. In the example shown in this figure, a tiara is selected as the accessory and a full-body costume is selected as the costume type. The generated image display area G15 displays a composite image in which the subject from the subject image is wearing the selected costume and accessories. At this time, an area for accepting the selection of items with different colors may be displayed on the fitting screen G10, or voice input instructions for changing colors may be accepted. The rental fees for the selected items may also be displayed.

[0058] ● Processing Flow Here, using Figure 6, we will explain an example of a processing flow for obtaining a composite image. First, for example, the user terminal 30 has been logged in by the user. Also, the display unit 31 of the user terminal 30 displays a fitting screen G10 as shown in Figures 2 and 3.

[0059] First, the subject image is set in the subject image setting field G11 (see Figure 2) by operation from the user terminal 30 (step S101). The image generation device 10 then refers to the scheduled loan dates of the items stored in the information storage device 40 and extracts items that can be loaned on the customer's desired loan date (step S102). Next, the extracted items are displayed on the user terminal 30. Note that steps S101 and steps S102 to S103 can be performed in any order and may be performed simultaneously.

[0060] Next, the user terminal 30 accepts the costume settings in the costume setting field G12 and the costume type setting field G14 (step S104). When the image generation device 10 instructs the artificial intelligence unit 20 to extract related items according to the set costume (step S105), the artificial intelligence unit 20 refers to the information storage device 40, extracts the related items, and outputs them to the image generation device 10. The image generation device 10 displays the output related items on the user terminal 30 (step S106). The user terminal 30 accepts the accessory settings in the accessory setting field G13 (step S107). Steps S104 to S106 may be repeated multiple times.

[0061] When the user terminal 30 receives a press of the start generation button G16 (step S108), the image generation device 10, via the AI ​​control unit 15, sends a composite image generation instruction to the artificial intelligence unit 20 (step S109). The artificial intelligence unit 20 generates a composite image using the first trained model and outputs it to the AI ​​control unit 15. The AI ​​control unit 15 receives this output, and the display control unit 11 presents the composite image received from the artificial intelligence unit 20 to the user terminal 30 (step S110).

[0062] As described above, the image generation device according to the present invention allows for easy and accurate capture of how a subject looks wearing various items. With this configuration, even customers who wish to try on a large number of items can be easily captured in terms of how they look when trying them on. For example, if the item a customer wishes to try on is not available at the store they are visiting, conventionally, it would be necessary to transport the item. However, with this image generation device, the need to transport items is eliminated, significantly reducing costs and time.

[0063] Furthermore, since you can select and try on costumes and accessories individually to generate images, you can check a wide range of combinations. This is especially useful when the costume is a kimono and the accessory is an obi (sash).

[0064] Although the present invention has been described above using embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes are possible within the scope of its gist. That is, although the above examples mainly described an example of virtually trying on wedding attire, the image generation device according to the present invention is applicable to examples not limited to such examples. Furthermore, in any of the application examples described below, the embodiments described in any other application example can be appropriately adopted.

[0065] For example, the image generation device according to the present invention can also be applied to try-on situations when customers purchase items in stores or on websites. In this case, for example, if a customer tries on clothes and then selects and purchases accessories such as necklaces or bags, the AI ​​control unit 15 may send a related item extraction instruction to the artificial intelligence unit 20, for example, to extract related items that match the clothes selected by the customer from the item database DB 42. The information storage device 40 also stores an appropriate database in which the store's inventory information is associated with item identification information, and the item extraction unit 13 may extract items in stock from the item database DB 42 and display them in the item selection field for selection. The AI ​​control unit 15 may also send an instruction to extract related items by referring to information on products that the store wants to promote. Furthermore, the AI ​​control unit 15 may refer to sales history and send an instruction to extract items that are presumed to have a high probability of selling as related items. Furthermore, the AI ​​control unit 15 may refer to the customer's purchase history and send an instruction to extract related items considering trends. The AI ​​control unit 15 may send an instruction to extract clothing that suits the customer from the item database DB42 based on the subject image.

[0066] Furthermore, the AI ​​control unit 15 may send instructions to the artificial intelligence unit 20 to generate a question sentence that asks the customer or store clerk for the information necessary for item extraction. The generated question sentence is displayed via the user terminal 30. The image generation device 10 also accepts responses from the customer or store clerk in text or voice. For example, the image generation device 10 accepts a change in the currently set costume color via voice input. With this configuration, it is even more convenient because the user can extract items while conversing with the image generation device 10.

[0067] Furthermore, the image generation device according to the present invention can also be applied to beauty salons, barbershops, and the like. In this case, the item database DB42 stores hairstyles that can be achieved in beauty salons, etc., as items. The image generation device 10 outputs a composite image of the subject with the selected hairstyle. In addition, hairstyles may be linked and stored with barbers and beauticians who are skilled in performing that hairstyle. With such a configuration, barbers and beauticians can share the image of the treatment with the customer. Also, customers can easily find a hairstyle that suits them.

[0068] Furthermore, the image generation device according to the present invention can also be applied to interior design shops and the like. In this case, the subject is the room in which the customer is considering the arrangement of furniture. That is, the image receiving unit 12 receives, for example, a subject image in which the customer's room is included as the subject. The image receiving unit 12 may also receive an image of the furniture item and its type. With such a configuration, the customer can easily see how the furniture will look when placed in their home.

[0069] Similar to the example described earlier, the AI ​​control unit 15 instructs the artificial intelligence unit 20 to determine the relative position of the item in the composite image relative to the subject, taking into account the item type, when issuing a composite image generation instruction. That is, an appropriate composite image can be generated in which the relative position to the room is defined according to the item type, such as placing an item of type sofa in the center of the room, or a ceiling light on the ceiling. The AI ​​control unit 15 may also place the item considering the type and position of furniture shown in the subject image. For example, the item may be placed near furniture of a type that is compatible with the item to be placed. Furthermore, if there is furniture of the same type as the item to be placed in the subject image, it may be possible to issue an instruction to generate a composite image in which the item is placed in place of that furniture, in response to appropriate instructions from the user terminal 30 or the like.

[0070] In this case, the item database DB42 stores furniture and other items available at the interior shop, along with their types. The information storage device 40 stores an appropriate database in which the store's inventory information is associated with item identification information. The item extraction unit 13 may extract items that are in stock from the item database DB42 and display them in the item selection field for selection. Furthermore, similar to the example described earlier, the AI ​​control unit 15 may send an instruction to the artificial intelligence unit 20 to extract related items from the item database DB42 that are compatible with the customer's room or the selected items, or it may send an instruction to extract related items by referring to information on products that the store wants to promote. Furthermore, the AI ​​control unit 15 may also send an instruction to extract related items after referring to sales history or customer purchase history. In addition, it may accept changes to the color, placement position, or orientation of the synthesized items via text or voice input. This configuration also allows for a simple and accurate understanding of how the subject is combined with various items. [Explanation of symbols]

[0071] 1. Image generation system 10 Image generation device 11 Display Control Unit 12 Image Reception Section 13 Item Extraction Section 14 Item Reception Department 15 AI control section 16 Communication Processing Unit 20 Artificial Intelligence Department 30 User terminals 40 Information storage device DB41 Customer Database DB42 Item Database DB43 Reservation Database

Claims

1. An image generation device that generates a composite image in which a subject included in a subject image is combined with a predetermined item, An image receiving unit that receives the aforementioned subject image, The aforementioned items are stored in a storage unit in advance, linked to loan reservation information. An item extraction unit refers to the loan reservation information of the aforementioned items and extracts the aforementioned items that are available for loan on the desired loan date. A display control unit that displays only the items extracted by the item extraction unit in the item selection field so that they can be selected, An item receiving unit that receives the setting of an item image of the item to be composited onto the subject image, An AI control unit transmits a composite image generation instruction to an artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item. It has, The item receiving unit accepts the setting of the item based on the selection in the item selection field. The AI ​​control unit, in the synthesized image generation instruction, instructs the artificial intelligence unit to determine the relative position of the item in the synthesized image with respect to the subject, taking into consideration the item type. Image generation device.

2. The subject is a person, The item receiving unit is capable of receiving the item image of the item type being a hairstyle. When the AI ​​control unit receives an item image of the item type being a hairstyle, it transmits a hairstyle synthesis instruction to the artificial intelligence unit to generate a composite image in which the hairstyle of the subject included in the subject image becomes the hairstyle included in the item image. The image generation apparatus according to claim 1.

3. The aforementioned items are pre-stored in a memory unit, linked to an index indicating the degree to which selection is recommended. The AI ​​control unit sends a recommended item extraction instruction to the artificial intelligence unit, which outputs a recommended item that is recommended for selection. The display control unit displays the items extracted by the recommended item extraction instruction in the item selection field. The image generation apparatus according to claim 1.

4. An image generation device that generates a composite image in which a subject included in a subject image is combined with a predetermined item, An image receiving unit that receives the aforementioned subject image, The aforementioned items are stored in a memory unit in advance, linked to inventory information, and an item extraction unit retrieves the items that are in stock by referring to the inventory information of the aforementioned items. A display control unit that displays only the items extracted by the item extraction unit in the item selection field so that they can be selected, An item receiving unit that receives the setting of an item image of the item to be composited onto the subject image, An AI control unit transmits a composite image generation instruction to an artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item. It has, The AI ​​control unit, in the synthesized image generation instruction, instructs the artificial intelligence unit to determine the relative position of the item in the synthesized image with respect to the subject, taking into consideration the item type. The aforementioned items are stored in the memory unit in advance, linked to an index of recommendation level for which selection is recommended, and the AI ​​control unit sends a recommendation item extraction instruction to the artificial intelligence unit to output a recommended item for which selection is recommended. The display control unit displays the items extracted by the recommended item extraction instruction in the item selection field. Image generation device.

5. The AI ​​control unit transmits a related item extraction instruction to the artificial intelligence unit, which outputs related items related to the item, based on the item and item type set by the item receiving unit. The display control unit displays the extracted related items in the item selection field. The image generating apparatus according to claim 1 or 4.

6. An image generation method that generates a composite image in which a subject included in a subject image is combined with a predetermined item, By computer, An image reception step for receiving the aforementioned subject image, The aforementioned items are stored in a memory unit in advance, linked to loan reservation information. The item extraction step involves referencing the loan reservation information of the aforementioned items and extracting the aforementioned items that are available for loan on the desired loan date. A display control step that makes only the items extracted by the item extraction step selectable in the item selection field, An item reception step that receives the setting of an item image of the item to be composited onto the subject image, An AI control step of transmitting a composite image generation instruction to the artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, Execute, In the item acceptance step, the item settings are accepted by the selection in the item selection field. In the AI ​​control step, the artificial intelligence unit is instructed to determine the relative position of the item in the composite image with respect to the subject, taking into consideration the item type. Image generation method.

7. An image generation method for generating a composite image in which a subject included in a subject image is combined with a predetermined item, An image reception step for receiving the aforementioned subject image, The aforementioned items are stored in a memory unit in advance, linked to inventory information, and the item extraction step involves referencing the inventory information of the aforementioned items and extracting the items that are in stock. A display control step that makes only the items extracted by the item extraction step selectable in the item selection field, An item reception step that receives the setting of an item image of the item to be composited onto the subject image, An AI control step of transmitting a composite image generation instruction to the artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, It has, In the AI ​​control step, the artificial intelligence unit is instructed to determine the relative position of the item in the composite image with respect to the subject, taking into consideration the item type. The aforementioned items are stored in the memory unit in advance, linked to an index of recommendation level for which selection is recommended, and in the AI ​​control step, a recommendation item extraction instruction is sent to the artificial intelligence unit to output a recommended item for which selection is recommended. In the display control step, the items extracted by the recommended item extraction instruction are displayed in the item selection field. Image generation method.

8. An image generation program that generates a composite image in which a subject included in a subject image is combined with a predetermined item, On the computer, An image reception step for receiving the aforementioned subject image, The aforementioned items are stored in a memory unit in advance, linked to loan reservation information. The item extraction step involves referencing the loan reservation information of the aforementioned items and extracting the aforementioned items that are available for loan on the desired loan date. A display control step that makes only the items extracted by the item extraction step selectable in the item selection field, An item reception step that receives the setting of an item image of the item to be composited onto the subject image, An AI control step of transmitting a composite image generation instruction to the artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, Make it run, In the item acceptance step, the item settings are accepted by the selection in the item selection field. In the AI ​​control step, the artificial intelligence unit is instructed to determine the relative position of the item in the composite image with respect to the subject, taking into consideration the item type. Image generation program.

9. An image generation program that generates a composite image in which a subject included in a subject image is combined with a predetermined item, On the computer, An image reception step for receiving the aforementioned subject image, The aforementioned items are stored in a memory unit in advance, linked to inventory information, and the item extraction step involves referencing the inventory information of the aforementioned items and extracting the items that are in stock. A display control step that makes only the items extracted by the item extraction step selectable in the item selection field, An item reception step that receives the setting of an item image of the item to be composited onto the subject image, An AI control step of transmitting a composite image generation instruction to the artificial intelligence unit to generate the composite image based on the subject image, the item image, and the item type of the item, Make it run, In the AI ​​control step, the artificial intelligence unit is instructed to determine the relative position of the item in the composite image with respect to the subject, taking into consideration the item type. The aforementioned items are stored in the memory unit in advance, linked to an index of recommendation level for which selection is recommended, and in the AI ​​control step, a recommendation item extraction instruction is sent to the artificial intelligence unit to output a recommended item for which selection is recommended. In the display control step, the items extracted by the recommended item extraction instruction are displayed in the item selection field. Image generation program.

Images