Virtual try-on system, virtual try-on method, and program

Abstract

To provide a virtual try-on system that enables virtual try-on of articles in various wearing modes.SOLUTION: A virtual try-on system includes: person image acquisition means for acquiring a person image of a person performing virtual try-on; item identification means for identifying a first item worn by the person based on the person image; item reception means for receiving designation of a second item to be performed virtual try-on; wearing mode reception means for receiving designation of a wearing mode for the second item; and output means for outputting a wearing image that shows the appearance of the second item in the designated wearing mode and makes the person appear to be wearing the second item, based on the person's body and the first item identified from the person image, and the designated second item.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to a virtual try-on system and the like. [Background technology]

[0002] Stores selling fashion items such as clothing and accessories are equipped with mirrors. These mirrors allow customers to hold products up to their bodies and judge whether they look good on them. In recent years, virtual try-on systems that utilize digital signage and augmented reality have become popular. Virtual try-on systems allow customers to easily try on a wide variety of products and color variations.

[0003] As an example of a system that allows virtual fitting, Patent Document 1 discloses a system in which a half mirror is provided on the display surface side of a video display board. In Patent Document 1, the video display board displays an image of clothing superimposed on a mirror image of the user reflected in the half mirror. Patent Document 1 further discloses that the video display board displays a combination of the user's image and the clothing image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 090345 Summary of the Invention [Problem to be solved by the invention]

[0005] Some articles can be worn in various ways. For example, a shirt can be worn buttoned up or unbuttoned and worn over other clothing. Patent Document 1 does not disclose how to wear an article in various ways. Therefore, only one virtual try-on is possible, making it difficult to determine whether the article suits a person.

[0006] The present disclosure aims to provide a virtual try-on system that makes it easy to determine whether an item suits a person. [Means for solving the problem]

[0007] The virtual try-on system according to the present disclosure comprises a person image acquisition means for acquiring a person image of a person performing the virtual try-on; an item reception means for receiving a designation of an item; a wearing mode reception means for receiving a selection of a wearing mode for the item; an item image selection means for selecting an item image showing a selected wearing mode from a plurality of item images each showing a different wearing mode for the item; and an output means for outputting an output image including a wearing image showing the item when worn by the person in the selected wearing mode, based on the person image and the selected item image.

[0008] The virtual try-on method of the present disclosure acquires a person image of a person performing the virtual try-on, accepts the specification of an item, accepts a selection of how the item will be worn, selects an item image showing the selected wearing manner from a plurality of item images each showing a different wearing manner of the item, and outputs an output image including a wearing image showing the item when worn by the person in the selected wearing manner based on the person image and the selected item image.

[0009] A program according to the present disclosure causes a computer to execute the following processes: acquire a person image of a person performing a virtual try-on; accept designation of an item; accept selection of a wearing manner of the item; select an item image showing the selected wearing manner from a plurality of article images each showing a different wearing manner of the item; and output an output image including a wearing image showing the item when worn by the person in the selected wearing manner based on the person image and the selected article image. The program may be stored in a computer-readable non-transitory recording medium. [Effects of the Invention]

[0010] According to the present disclosure, it is easy to determine whether an article suits a person. [Brief explanation of the drawings]

[0011] [Figure 1] 1 is a block diagram showing an example of the configuration of a virtual try-on system according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of an article image. [Figure 3] FIG. 10 is a diagram showing an example of a person image. [Figure 4] FIG. 10 is a diagram illustrating an example of an output image displayed on a smart mirror. [Figure 5] 10 is a flowchart showing an example of the operation of the virtual try-on system. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of a virtual try-on system according to a second embodiment. [Figure 7] FIG. 10 is a diagram showing an example of a screen including an output image. [Figure 8] FIG. 5 is a block diagram showing an example of the hardware configuration of a computer 500. DETAILED DESCRIPTION OF THE INVENTION

[0012] [First embodiment] The virtual try-on system 100 according to the first embodiment outputs to the smart mirror a wearing image showing an article when worn in a certain manner by a person standing in front of the smart mirror. This allows the person to look at the smart mirror and try on the article in various ways. In the following example, a smart mirror installed in a store will be described. However, the installation location of the smart mirror is not particularly limited.

[0013] 1 is a block diagram showing an example of the configuration of a virtual try-on system 100 according to the first embodiment. The virtual try-on system 100 includes a person image acquisition unit 101, an item reception unit 102, a wearing style reception unit 103, an item image selection unit 104, and an output unit 105.

[0014] The virtual try-on system 100 is also communicatively connected to the camera 10 and smart mirror 20. The virtual try-on system 100 may further be communicatively connected to an article image DB (Database) 30. The virtual try-on system 100 may be directly connected to the camera 10, smart mirror 20, and article image DB 30 via wires or may be connected via a network. The virtual try-on system 100 may further be connected to the Internet.

[0015] The camera 10 captures a person image of a person in front of the smart mirror 20. Therefore, the camera 10 is installed near the smart mirror 20. The camera 10 may be formed integrally with the smart mirror 20.

[0016] The smart mirror 20 has both a mirror function and a display function. The smart mirror 20 is also called a mirror display. The smart mirror 20 is realized, for example, by placing a mirror that partially transmits light in front of a display. Since the image projected by the display passes through the mirror, a person standing in front of the smart mirror 20 can see the image displayed by the display. In areas where the display does not project an image, the person can see the mirror image reflected by the mirror.

[0017] The smart mirror 20 may be equipped with an operation unit 22 so that a person can operate the smart mirror 20 as needed. Below, we will explain a case where the smart mirror 20 is equipped with a touch operation panel as the operation unit 22. However, the form of the operation unit 22 is not limited to this. The smart mirror 20 may be connected to an operation terminal such as a tablet or smartphone via a wired or wireless connection as the operation unit 22. The smart mirror 20 may accept operations from these operation terminals.

[0018] The article image DB30 is a database that stores article images. The article image DB30 stores a plurality of article images each showing a different manner of wearing each article. Here, the article is not particularly limited as long as it is something worn by a person. The article includes, for example, clothes, bags, hats, and accessories.

[0019] Each wearing mode is a way of wearing a single article in a different arrangement. Examples of different wearing modes include, for example, wearing an article front to back or inside out, or wearing it in a different position on the body. The wearing mode of an article can also be changed by changing the number of buttons that are unbuttoned, the way a ribbon is tied, or the width of the hem that is rolled up. Furthermore, when an article is worn in combination with other articles, the wearing mode of the article can also be changed by changing the combination of the articles or the order in which the articles are layered.

[0020] The apparent shape of an article when worn in one wearing manner differs from the apparent shape when worn in another wearing manner, and therefore the article image DB30 stores article images for each different wearing manner.

[0021] The item images stored in the item image DB 30 are, for example, images showing how items sold in a store or items sold in the past are worn. In this case, the store operator may register the item images. However, the item images are not limited to this. The registered item images may also be images of items photographed by a user. The registered item images may also be generated from images of items photographed by a user. The case where an item is photographed by a user will be described later.

[0022] Fig. 2 is a diagram showing an example of an article image. The article image in Fig. 2 shows a wearing style in which the shirt is worn with all front buttons open and as the top layer of layered clothing. In addition to the article image in Fig. 2, the article image DB30 may store article images showing other wearing styles for the same shirt. For example, an article image showing a wearing style with the buttons fastened, an article image showing a wearing style with the sleeves rolled up, and an article image showing a wearing style worn under other clothes may be stored.

[0023] The person image acquisition unit 101 acquires a person image. For example, the person image acquisition unit 101 acquires a person image obtained by capturing a person in front of the smart mirror 20 from the camera 10. From the person image, it is possible to recognize a part of the person's body or the entire body appearance. The part of the body recognized from the image is, for example, the front, back, face, upper body, lower body, feet, etc., and can be changed appropriately depending on the installation positions of the smart mirror 20 and the camera 10. Furthermore, from the person image, it is possible to recognize the position of the person relative to the smart mirror 20. Furthermore, it may be possible to recognize items worn by the person from the person image.

[0024] 3 is a diagram showing an example of a person image, which includes the entire front view of the person.

[0025] The item receiving unit 102 receives the designation of an item. For example, the item receiving unit 102 may receive a designation of an item to virtually try on based on a user's operation. The user performs an operation on the operation unit 22 to designate an item to virtually try on from among the item options displayed on the smart mirror 20 or the operation terminal. The item receiving unit 102 receives the designation of an item from the operation unit 22.

[0026] The wearing mode receiving unit 103 receives a selection of a wearing mode of the article. For example, the wearing mode receiving unit 103 may receive the selection of a wearing mode based on a user's operation. The user performs an operation on the operation unit 22 to select a wearing mode for virtual try-on from among options of wearing modes displayed on the smart mirror 20 or the operation terminal. The wearing mode receiving unit 103 receives the selection of a wearing mode from the operation unit 22.

[0027] The article image selection unit 104 selects an article image showing the selected wearing manner from a plurality of article images each showing a different wearing manner of an article. For example, the article image selection unit 104 selects an article image corresponding to the wearing manner accepted by the wearing manner acceptance unit 103 from a plurality of article images stored in the article image DB 30.

[0028] The item image selection unit 104 may select an item image searched through the Internet. For example, the item image selection unit 104 obtains an item image of a specified item from the Internet by image search. Alternatively, the item image selection unit 104 may obtain and select an item image generated by another image generation device (not shown). For example, the image generation device generates a new item image showing a different wearing manner based on an item image showing one wearing manner.

[0029] The item image selection unit 104 may select an item image from among item images including item images pre-stored in the item image DB 30, item images searched from the Internet, or generated item images. If there is no pre-registered item image showing the selected wearing manner, the item image selection unit 104 may select an item image from among item images including item images searched from the Internet or generated item images.

[0030] The output unit 105 outputs an output image based on the person image acquired by the person image acquisition unit 101 and the article image selected by the article image selection unit 104. For example, the output unit 105 outputs an output image including a wearing image showing an article when worn by a person in the selected wearing manner. At this time, the output unit 105 outputs the wearing image by aligning the position of the body in the mirror image seen by the person with the position of the displayed article based on the position of the person identified from the person image.

[0031] The output unit 105 may output the article image as a wearing image. For example, the output unit 105 outputs the article image as a wearing image by displaying the article image in a position that matches the person's body. Alternatively, the output unit 105 may generate and output a wearing image from the article image. For example, the output unit 105 may output a wearing image that is generated by processing the article image to match the body shape and posture of the person identified from the person image. The posture of the person includes the person's way of standing, pose, or body orientation in relation to the mirror.

[0032] FIG. 4 is a diagram showing an example of an output image displayed on the smart mirror 20. The smart mirror 20 reflects a mirror image of the person in front of the smart mirror 20. The output unit 105 outputs an image of the person wearing the shirt as the output image. The output unit 105 displays the wearing image according to the person's position and body shape, so that the person sees an image of the person wearing the item.

[0033] The output unit 105 may further output a selection of items. The selection of items is represented by text or an image. The output unit 105 outputs the selection of items to the smart mirror 20 or the operation terminal that is the operation unit 22.

[0034] The output unit 105 may output multiple types, sizes, or color variations of products as options. The output unit 105 may output products represented by product images stored in the product image DB 30 as options. The output unit 105 may output products currently sold in stores or products previously sold as product options.

[0035] The output unit 105 may output an article image showing the article for which designation has been accepted, in addition to the wearing image.

[0036] Furthermore, the output unit 105 may output options for the wearing mode for the item whose designation has been accepted by the item accepting unit 102. The options for the wearing mode may be displayed using characters representing the wearing mode. Alternatively, the options for the wearing mode may be displayed by arranging item images representing the respective wearing modes. The output unit 105 outputs the options for the wearing mode to the smart mirror 20 or to an operation terminal that is the operation unit 22.

[0037] The options for the manner of wearing may be predetermined depending on what manner of wearing is possible for each article or each type of article.

[0038] Furthermore, the output unit 105 may output options for the wearing mode of the specified item based on the wearing mode indicated by the item image that the item image selection unit 104 can acquire. The output unit 105 may determine whether the item image selection unit 104 can acquire an item image. The item images that the item image selection unit 104 can acquire include item images stored in the item image DB 30, item images searched via the Internet, and item images generated based on one item image. For example, the output unit 105 outputs options for the wearing mode of the specified item that correspond to the wearing mode indicated by the item image stored in the item image DB 30. The output unit 105 does not display options for wearing modes for which no article image is stored, or grays them out to prevent selection.

[0039] The output unit 105 outputs options based on the wearing modes indicated by the obtainable article images, so that only options corresponding to the wearing modes for which the output unit 105 can output wearing images are output. Therefore, the user can select a wearing mode from among the wearing modes for which a natural-looking wearing image can be output.

[0040] In FIG. 4, the smart mirror 20 displays article images representing two wearing modes as options for wearing modes. The user selects a wearing mode via the operation unit 22, such as a touch operation panel. The smart mirror 20 may display the selected wearing mode differently from other wearing modes based on control by the output unit 105. For example, as shown in FIG. 4, the smart mirror 20 displays a frame surrounding the selected wearing mode.

[0041] In FIG. 4, an example has been described in which a virtual try-on is performed for one shirt. However, virtual try-on may be performed for multiple items simultaneously. In this case, the item receiving unit 102 receives the designation of multiple items. The wearing style receiving unit 103 may receive the selection of a wearing style for each item. The item image selecting unit 104 selects an item image corresponding to the wearing style of each item. The output unit 105 outputs an output image including wearing images of the multiple items based on the selected multiple item images.

[0042] 5 is a flowchart showing an example of the operation of the virtual try-on system 100. The virtual try-on system 100 may start the operation of FIG. 5 in response to, for example, a person being photographed in front of the smart mirror 20.

[0043] The person image acquisition unit 101 acquires a person image (step S1). The item reception unit 102 receives a designation of an item (step S2). The wearing style reception unit 103 receives a selection of a wearing style of the item (step S3).

[0044] The article image selection unit 104 selects an article image showing the selected wearing manner from among a plurality of article images each showing a different wearing manner of an article (step S4).

[0045] The output unit 105 outputs an output image including a wearing image showing the item when worn by a person in the selected wearing manner, based on the acquired person image and the selected item image (step S5).

[0046] After step S5, steps S3 to S5 may be repeated. For example, the wearing style receiving unit 103 determines whether another wearing style has been selected (step S6). If another wearing style has been selected (step S6: Yes), the wearing style receiving unit 103 receives the selection of the other wearing style (step S3). The article image selecting unit 104 selects an article image showing the selected wearing style (step S4). Then, the output unit 105 outputs a wearing image showing a person wearing the article in the selected other wearing style, instead of the original wearing image (step S5). If another wearing style has not been selected (step S6: No), the virtual try-on system 100 ends the operation of FIG. 5.

[0047] After step S5, the item receiving unit 102 may accept the designation of another item. If another item is designated, steps S2 to S5 are repeated. The item receiving unit 102 may accept the designation of the other item as an additional item. Alternatively, the item receiving unit 102 may cancel the designation of the previously designated item and accept the designation of the other item.

[0048] After step S5, the item receiving unit 102 may also receive a request to delete an item.

[0049] According to the first embodiment, the person image acquisition unit 101 acquires a person image, the item acceptance unit 102 accepts the designation of an item, and the wearing style acceptance unit 103 accepts the selection of the wearing style of the item. Then, the item image selection unit 104 selects an item image showing the selected wearing style from a plurality of item images each showing a different wearing style of the item. Furthermore, the output unit 105 outputs an output image including a wearing image showing the item when worn by a person in the selected wearing style based on the acquired person image and the selected item image. This allows for virtual try-on of the item in various wearing styles. This makes it easy to determine whether the item suits the person undergoing the virtual try-on.

[0050] In virtual try-on, when only one wearing style can be tried on, it is not guaranteed that the person will wear the item in that wearing style. According to the first embodiment, virtual try-on is possible in a wearing style that is thought to be the way a person will actually wear it, and it is possible to support the judgment of whether the item suits the person. Furthermore, there are cases where it is not possible to judge whether the item suits the person by trying on only one wearing style. According to the first embodiment, it is possible to support the judgment of whether the item suits the person by looking at various wearing styles.

[0051] Furthermore, the article image selection unit 104 selects an article image showing the selected wearing manner from a plurality of article images each showing a different wearing manner of an article, and the output unit 105 outputs an output image based on the selected article image. Therefore, the output unit 105 can output an output image showing the appearance of an article that differs depending on the wearing manner. Therefore, according to the first embodiment, it is possible to present to a person the appearance of an article in various wearing manners in a more understandable manner than when an article image is not selected for each wearing manner.

[0052] By introducing the virtual try-on system 100 according to the first embodiment for virtual try-on in a store, customers can easily try on items, including merchandise. Furthermore, customers can see how an item looks when worn in various arrangements. Therefore, according to the first embodiment, customers can be encouraged to purchase the product.

[0053] [Second embodiment] The virtual try-on system 200 according to the second embodiment outputs to a display a wearing image showing an article when worn by a person in a certain wearing manner, allowing a user viewing the display to try on various wearing manners of the article.

[0054] Fig. 6 is a block diagram showing an example of the configuration of a virtual try-on system 200 according to the second embodiment. Regarding the configuration of the virtual try-on system 200, the description of the same components as those of the virtual try-on system 100 according to the first embodiment will be omitted. In Fig. 6, the virtual try-on system 200 differs from the virtual try-on system 100 according to the first embodiment in that it is connected to a display 21 instead of a smart mirror 20.

[0055] The display 21 is not particularly limited as long as it allows the user to check the output image output by the output unit 105. For example, the display 21 may be a signage installed in a store. The smart mirror 20 according to the first embodiment can also be used as the display 21 in the second embodiment. The display 21 may also be a smartphone, a tablet, or a head-mounted display.

[0056] The virtual try-on system 200 is communicatively connected to the camera 10 and the operation unit 22 as necessary. The camera 10 and the operation unit 22 may be provided integrally with the display 21 or may be realized as separate devices. The camera 10 may be installed so as to capture a frontal image of a person facing the display 21, which is a signage, but the installation position is not limited to this. For example, the camera 10 may be installed so as to capture a backside image of a person facing the display 21.

[0057] The person image acquisition unit 101 acquires a person image of a person who will virtually try on clothes. The person image acquisition unit 101 acquires a person image in which the appearance of a part of the person's body or the entire body can be recognized.

[0058] In one example, the person image acquisition unit 101 acquires a person image from the camera 10. For example, the person image acquisition unit 101 acquires a person image obtained by capturing a person standing in front of the display 21, which is a signage. Alternatively, the person image acquisition unit 101 may acquire an image of a person captured by the camera 10 included in the display 21, which is a smartphone.

[0059] However, the specific method for acquiring the person image is not particularly limited. In another example, the person image acquisition unit 101 may acquire a person image that has been photographed in advance. That is, in the second embodiment, the camera 10 may be provided as needed.

[0060] Similar to the first embodiment, the output unit 105 outputs an output image including a wearing image to the display 21 based on the person image and the selected article image. However, in the second embodiment, the output unit 105 outputs an output image including a person image and a wearing image. For example, the output unit 105 outputs an output image in which a wearing image showing the article when worn by a person in the selected wearing manner is superimposed on the person image.

[0061] The output unit 105 may output an output image including a left-right inverted image of the person and an image of the person wearing the product to the display 21. As a result, the same image as the user sees himself in a mirror is output to the display 21. By outputting such an output image to the display 21, which is a signage installed in a store, the user can use the display 21 as a mirror.

[0062] The output unit 105 may output an output image depending on the orientation of the person in the person image. If the person image is an image capturing the back of the person, the output unit 105 outputs an output image showing the back view of the person wearing the article in the selected wearing manner. This allows the user to easily check, on the display 21, the back view of the person in the selected wearing manner, which cannot be seen in a mirror.

[0063] The virtual try-on system 200 may include an identification unit that identifies the orientation of a person in a person image. In this case, the output unit 105 outputs an output image based on the identification result of the person's orientation by the identification unit. The identification unit may, for example, identify the orientation of the legs or face from the person image to identify the person's orientation. The identification unit may also identify the orientation of a person in a person image based on the relationship between the installation positions of the camera 10 that captured the person image and the display 21, which is a signage. The identification unit identifies an image captured by the camera 10 installed in a position directly facing the display 21 as an image capturing the back of the person.

[0064] The output unit 105 may output an output image in which an article image is superimposed on a person image as a wearing image. For example, the output unit 105 outputs the article image as a wearing image by displaying the article image at a position that matches the body of the person image. Alternatively, the output unit 105 may output a wearing image in which the article image has been processed to match the person's body shape or posture, and superimpose it on the person image.

[0065] Fig. 7 is a diagram showing an example of a screen including an output image displayed on the display 21. For example, as shown in Fig. 7, the output unit 105 outputs an output image in which the article image in Fig. 2 is superimposed as a wearing image at a position that matches the body of the person image in Fig. 3.

[0066] The screen of Fig. 7 further includes an image showing the specified item. The screen of Fig. 7 also includes options for wearing styles. In Fig. 7, options for wearing styles with the buttons open and with the buttons closed are displayed, and a check box indicates that the wearing style with the buttons open has been selected. The display of the options is not limited to the above, and the options may be displayed in a pull-down menu, for example.

[0067] According to the second embodiment, similar to the first embodiment, virtual try-on of various ways of wearing an article is realized. Also, according to the second embodiment, the output unit 105 outputs an output image including an image of a person and an image of the article being worn. Therefore, the user can look at the display 21 and try on various ways of wearing the article.

[0068] When the display 21 is installed in a store, the second embodiment can also encourage customers to purchase products, as in the first embodiment. Furthermore, when a user owns the display 21, the user can try on various ways of wearing the product at any location, such as at home.

[0069] [Variations] The virtual try-on systems 100 and 200 according to the respective embodiments can be modified as follows, for example.

[0070] In one variation, the virtual try-on system 100, 200 may further include an identification unit that identifies the user whose person image is acquired. In this case, the item image DB 30 stores item images of items owned by the user. For example, the item image DB 30 stores item images for each user. The item receiving unit 102 extracts items owned by the user identified by the identification unit from the item image DB 30. The item receiving unit 102 outputs the extracted items to the output unit 105 as item options. In this way, the item receiving unit 102 may accept a designation of an item from among the items owned by the user. Identification of a person is not particularly limited, and may be performed, for example, by facial recognition using a person's image. Identification of a person may also be performed using a membership code carried by the person.

[0071] When the item receiving unit 102 receives the designation of an item owned by the person whose person image was acquired, the output unit 105 can output an image of the person wearing the item in the selected manner. For example, if a person is actually trying on clothes sold in a store, the output unit 105 can output an image of the person wearing the clothes that the person keeps at home. The output unit 105 can also output an output image that includes an image of the sold item being worn and an image of the clothes that the person keeps at home. In this way, the virtual try-on systems 100, 200 can present to customers how the clothes sold in the store will look when combined with the clothes that the person keeps at home.

[0072] In one modified example, the item receiving unit 102 may receive the designation of an item based on the person image acquired by the person image acquiring unit 101.

[0073] The item receiving unit 102 may accept the designation of an item according to the range of the body identified from the person image. In this case, the virtual try-on systems 100 and 200 may further include an identification unit that identifies which part of the person is captured in the person image. For example, the item receiving unit 102 may stop accepting the designation of an item to be worn on a part of the person image that the identification unit has not identified. If the person image is an image of the person's upper body, the identification unit does not identify the person's feet. Therefore, the item receiving unit 102 may stop accepting the designation of shoes. The item receiving unit 102 may output a warning to the output unit 105 when an item to be worn on a part of the body other than the part identified by the identification unit is specified. Furthermore, the item receiving unit 102 may exclude the item for which acceptance is to be stopped from the options and cause the output unit 105 to output the item options. Alternatively, the item receiving unit 102 may gray out the item for which acceptance is to be stopped and cause the output unit 105 to output the item options.

[0074] The virtual try-on systems 100 and 200 may further include an identification unit that identifies an item actually worn by a person through image recognition of the person's image. The identification unit may identify the item by comparing the person's image with pre-registered item images. The item receiving unit 102 then accepts the designation of the item, regarding the identified item as the designated item. The item receiving unit 102 may output the identified item to the output unit 105 as an item option.

[0075] The virtual try-on systems 100, 200 may further include an identification unit that identifies the actual wearing style of an article worn by a person based on the person's image. The wearing style receiving unit 103 may then exclude the wearing style actually worn by the person from the options and cause the output unit 105 to output the wearing style options. Alternatively, the wearing style receiving unit 103 may gray out the wearing style actually worn by the person and cause the output unit 105 to output the wearing style options. In this way, the wearing style receiving unit 103 receives the selection of a wearing style different from the wearing style actually worn by the person.

[0076] In this way, the item receiving unit 102 receives the designation of an item actually worn by a person, and the output unit 105 can output images of the item being worn by the person in different wearing styles. Therefore, for an item that a person is already wearing, it is possible to easily try out different wearing styles without having to put on or take off the item or open and close the buttons of the item.

[0077] The item receiving unit 102 may accept the designation of multiple items, including items actually worn by the person performing the virtual try-on, and items not worn by the person. For example, if the person is actually wearing a shirt with the buttons fastened, the person may want to check how the shirt looks when the buttons are open and other items are worn underneath. In this case, the output unit 105 may output output images including both a wearing image of the unworn item and a wearing image of the worn item. In this case, the output unit 105 may output a wearing image showing a wearing manner different from the wearing manner of the item actually worn. For example, if the person is wearing a shirt with the buttons fastened, the output unit 105 outputs a wearing image of the item with the buttons open. Therefore, when the person is wearing an item that is not actually worn, the user can easily try out different looks when wearing the item that they are actually wearing in different ways.

[0078] The item receiving unit 102 may also accept the designation of an item, with the designated item being an item recommended to be worn in combination with an item currently worn by the person. In this case, the virtual try-on systems 100 and 200 may further include an identification unit that identifies the item worn by the person from the person's image, and a determination unit that determines the recommended item. The determination unit determines, using any method, an item recommended to be worn in combination with the identified item. The determination unit may determine the recommended item from among items owned by the user at home. For example, the determination unit may refer to a database that pre-registers recommended item combinations. The determination unit may also use AI (artificial intelligence) to determine an item that goes well with the currently worn item based on the shape and color of the item.

[0079] The item receiving unit 102 may accept the designation of an item by treating the determined item as the designated item based on the determination result by the determination unit. Alternatively, the item receiving unit 102 may accept the designation of an item based on a user's selection from among the items determined by the determination unit. The output unit 105 may output, as an option, an item determined to be recommended for wearing in combination with the identified item. For example, the output unit 105 outputs an output image including a wearing image of the designated item when worn in the selected wearing manner. Then, the item receiving unit 102 accepts the designation of an item selected from the options.

[0080] The wearing style receiving unit 103 may receive a selection of a wearing style that is recommended to be worn in combination with an item worn by a person. The output unit 105 may output the recommended wearing style as an option. The recommended wearing style may be predetermined according to the combination of items.

[0081] In one variation, the item receiving unit 102 may accept the designation of an item based on the results of reading a tag attached to the item. In this case, a tag reader is communicatively connected to the smart mirror 20 or the display 21. The tag attached to the item is not particularly limited as long as it identifies the item, and may be, for example, a tag on which a code including a barcode and a two-dimensional code is printed, or an RF (Radio Frequency) tag. The item receiving unit 102 accepts the designation of an item based on the results of reading by the tag reader. By having the item receiving unit 102 accept the designation of an item based on the results of reading the tag attached to the item, the user can easily specify the item to virtually try on.

[0082] When a plurality of items are image-recognized from a person image and when tags of a plurality of items are read by the above method, the output unit 105 may output the plurality of items as item options. The item receiving unit 102 receives the designation of an item selected by a user operation.

[0083] In one variation, the virtual try-on systems 100 and 200 may further include an image generation unit that generates an article image showing another wearing manner from an article image showing one wearing manner. The image generation unit generates the article image by processing article images stored in the article image DB 30 or article images searched via the Internet. The image generation unit may generate a 3D model image as the article image from an image acquired by any method. For example, the image generation unit generates an article image showing an article worn with the buttons open from an article image showing an article worn with the buttons fastened. If there is no pre-registered article image showing the selected wearing manner, the image generation unit may generate an article image showing the selected wearing manner.

[0084] When the item receiving unit 102 receives the designation of an item by specifying an item that is actually worn by a person as the designated item, the image generating unit may generate an item image based on the person image. That is, the image generating unit extracts an item image showing the manner in which the person actually wears the item from the person image. Then, the image generating unit generates an item image showing another manner in which the item is worn based on the extracted item image.

[0085] In this way, the item image selection unit 104 may select an item image from among item images including the item image generated by the image generation unit. The item image selection unit 104 may select an item image from two images: an item image extracted from a person image and an image generated by the image generation unit. By the image generation unit generating an item image, it becomes possible to output a wearing image even when there is no suitable item image in the item image DB 30 or on the Internet.

[0086] In the first embodiment, it has been described that an article image may be registered in the article image DB 30 based on an image taken by a user. An article image showing how an article is being worn, generated from an image of an article taken by a user, may be registered in the article image DB 30. For example, if an image of a person wearing an article is taken by a user, the image may be registered as an article image. Also, if an image of an article placed flat without being worn is taken by a user, an article image may be generated by processing the image.

[0087] Users take pictures of items they own and register the images. By registering images based on the images taken by the user, images of items that are not sold in stores are also stored. This allows users to virtually try on a variety of items.

[0088] However, when product images are registered based on images taken by a user, the number of images taken by the user may be insufficient, or the quality of the images may be insufficient. If the number or quality of the images taken by the user is insufficient, it may be difficult to generate product images, resulting in a shortage of product images.

[0089] Examples of cases where the number of images taken by the user is insufficient or the image quality is poor include cases where the taken images are too bright or too dark. Furthermore, when images of an article taken from multiple directions are required to generate an article image, if images are taken from only one direction, there may be insufficient article images. Furthermore, when an article image showing how the article is worn cannot be generated from a laid-down image, there may be insufficient article images if only a laid-down image is taken.

[0090] As described above, the item image selection unit 104 may select an item image acquired from the Internet when there are insufficient pre-registered item images showing the selected wearing manner. The item image selection unit 104 may also determine that there are insufficient pre-registered item images. The item image selection unit 104 may also acquire item images of the same item or items with similar appearances from the Internet by image search.

[0091] Incidentally, for a certain item, multiple item images showing the same type of wearing style may be registered in the item image DB 30. For example, when stylists or users register item images, multiple stylists may register item images showing the same type of wearing style. The more item images showing the same type of wearing style there are, the more likely that the wearing style is recommended.

[0092] Therefore, in one variation, the virtual try-on systems 100 and 200 may further include a determination unit that determines a recommended wearing style. The determination unit determines a recommended wearing style based on the number of article images showing the same type of wearing style for the specified article. For example, the determination unit determines that the wearing style with the most article images showing the same type of wearing style is the recommended wearing style.

[0093] The method by which the determination unit determines the recommended wearing style is not limited to the above. For example, the determination unit may determine the wearing style that suits a person based on the person's appearance. For example, the determination unit may determine the recommended wearing style based on the person's height.

[0094] The output unit 105 may output the option for the recommended wearing manner differently from the other options. The wearing manner receiving unit 103 receives the selection of the determined recommended wearing manner. This allows the output unit 105 to output a wearing image showing the recommended wearing manner.

[0095] Furthermore, the output unit 105 may output both the output image according to the first embodiment and the output image according to the second embodiment to the smart mirror 20. For example, the output unit 105 outputs the wearing image according to the first embodiment superimposed on the mirror image of the person reflected in the smart mirror 20 in order to show the appearance of the item on the side facing the smart mirror 20. Furthermore, the output unit 105 may output the wearing image according to the second embodiment and the person image, shifted from the mirror image of the person, in order to show the appearance of the item on the back side of the person, which is not reflected in the mirror of the smart mirror 20. This allows the user to check the appearance of the front and back sides at the same time.

[0096] [Hardware configuration] In the above-described embodiments, each component of the virtual try-on systems 100 and 200 represents a functional block. Some or all of the components of the virtual try-on systems 100 and 200 may be realized by any combination of a computer 500 and a program.

[0097] Fig. 8 is a block diagram showing an example of the hardware configuration of a computer 500. Referring to Fig. 8, the computer 500 includes, for example, a processor 501, a read only memory (ROM) 502, a random access memory (RAM) 503, a program 504, a storage device 505, a drive device 507, a communication interface 508, an input device 509, an input / output interface 511, and a bus 512.

[0098] The processor 501 controls the entire computer 500. The processor 501 may be, for example, a CPU (Central Processing Unit). The number of processors 501 is not particularly limited, and there may be one or more processors 501.

[0099] The program 504 includes instructions for realizing each function of the virtual try-on systems 100, 200. The program 504 is stored in advance in the ROM 502, RAM 503, or storage device 505. The processor 501 executes the instructions included in the program 504 to realize each function of the virtual try-on systems 100, 200. The RAM 503 may also store data to be processed in each function of the virtual try-on systems 100, 200.

[0100] The drive device 507 reads and writes data from and to the recording medium 506. The communication interface 508 provides an interface with a communication network. The input device 509 is, for example, a mouse or a keyboard, and accepts information input from a user or the like. The output device 510 is, for example, a display, and outputs (displays) information to a user or the like. The input / output interface 511 provides an interface with peripheral devices. The bus 512 connects these hardware components. The program 504 may be supplied to the processor 501 via a communication network, or may be stored in advance on the recording medium 506, read by the drive device 507, and supplied to the processor 501.

[0101] It should be noted that the hardware configuration shown in FIG. 8 is an example, and other components may be added, or some components may not be included.

[0102] There are various variations in the method of realizing the virtual try-on systems 100, 200. For example, the virtual try-on systems 100, 200 may be realized by any combination of different computers and programs for each component. Furthermore, multiple components included in the virtual try-on systems 100, 200 may be realized by any combination of a single computer and program.

[0103] Some or all of the components of the virtual try-on system 100 may be realized by the smart mirror 20 or the display 21. That is, a program that realizes the components of the virtual try-on system 100 may be installed on the computer of the smart mirror 20 or the display 21. For example, the person image acquisition unit 101 and the output unit 105 may be realized by the smart mirror 20 or the display 21. The remaining components may be realized by a server device separate from the smart mirror 20 or the display 21.

[0104] Furthermore, at least a portion of the virtual try-on systems 100 and 200 may be provided in a SaaS (Software as a Service) format. That is, at least a portion of the functions for realizing the virtual try-on systems 100 and 200 may be executed by software that runs over a network.

[0105] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, the configurations in the respective embodiments can be combined with each other without departing from the scope of the present disclosure.

[0106] This application claims priority based on Japanese Patent Application No. 2022-134306, filed on August 25, 2022, the disclosure of which is incorporated herein in its entirety.

[0107] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. [Appendix 1] a person image acquisition means for acquiring a person image of a person performing virtual fitting; an item receiving means for receiving a designation of an item; a wearing mode receiving means for receiving a selection of a wearing mode of the article; an article image selection means for selecting an article image showing a selected wearing manner from a plurality of article images each showing a different wearing manner of the article; an output means for outputting an output image including a wearing image showing the article when the person wears the article in the selected wearing manner, based on the person image and the selected article image; A virtual try-on system comprising: [Appendix 2] Each of the wearing modes is a way of wearing the article in a different arrangement. 1. A virtual try-on system as described in Appendix 1. [Appendix 3] The output means outputs options for a wearing style based on the wearing style indicated by the obtainable article image for the specified article. 3. The virtual try-on system according to claim 1 or 2. [Appendix 4] further comprising an image generating means for generating, from the article image showing one wearing manner, the article image showing another wearing manner; The product image selection means selects the generated product image. 3. The virtual try-on system according to claim 1 or 2. [Appendix 5] the item receiving means receives a designation of the item, with the item identified from the person image as the designated item; The wearing style receiving means receives a selection of a wearing style different from the wearing style of the article identified from the person image. 3. The virtual try-on system according to claim 1 or 2. [Appendix 6] a determining unit for determining an article that is recommended to be worn in combination with the article identified from the person image; The product receiving means receives the designation of the product, with the recommended product being the designated product. 3. The virtual try-on system according to claim 1 or 2. [Appendix 7] a determination means for determining a recommended wearing manner based on the number of the article images showing the same type of wearing manner of the article; The wearing mode receiving means receives a selection of the recommended wearing mode determined. 3. The virtual try-on system according to claim 1 or 2. [Appendix 8] the person image acquisition means acquires the person image of the person in front of the smart mirror; The output means outputs the output image to the smart mirror. 3. The virtual try-on system according to claim 1 or 2. [Appendix 9] Acquire a person image of a person who will be performing the virtual try-on; Accept the specification of the item, Accepting a selection of a wearing style of the article; selecting an article image showing a selected wearing manner from a plurality of article images each showing a different wearing manner of the article; Based on the person image and the selected article image, an output image including a wearing image showing the article when the person wears the article in the selected wearing manner is output. Virtual try-on method. [Appendix 10] Acquire a person image of a person who will be performing the virtual try-on; Accept the specification of the item, Accepting a selection of a wearing style of the article; selecting an article image showing a selected wearing manner from a plurality of article images each showing a different wearing manner of the article; Based on the person image and the selected article image, an output image including a wearing image showing the article when the person wears the article in the selected wearing manner is output. A program that causes a computer to perform a process. [Explanation of symbols]

[0108] 100, 200 Virtual try-on system 101 Person image acquisition unit 102 Goods Reception 103 Wearing style reception section 104 Item image selection section 105 Output section 10 Camera 20. Smart Mirror 21 Display 22 Control section 30 Product Image DB

Claims

1. a person image acquisition means for acquiring a person image of a person performing virtual fitting; an item identification means for identifying a first item worn by the person based on the person image; an item receiving means for receiving a designation of a second item for virtual try-on; a wearing mode receiving means for receiving a designation of a wearing mode of the second article; an output means for outputting a wearing image that shows the appearance of the second article in the specified wearing manner and makes the person appear to be wearing the second article, based on the body of the person and the first article identified from the person image and the specified second article; A virtual try-on system comprising:

2. further comprising a posture identification means for identifying a posture of the person based on the person image, The output means outputs a wearing image that makes the person appear to be wearing the second article in the identified posture of the person. The virtual try-on system according to claim 1 .

3. The identified posture of the person is at least one of the person's standing, pose, and body orientation relative to the camera. The virtual try-on system according to claim 2 .

4. the posture identification means identifies the orientation of the body relative to the camera; The output means outputs a wearing image showing an appearance of the second article on the back side of the person's body when the person faces the back side of the body toward the camera. The virtual try-on system according to claim 3 .

5. further comprising a body part identification means for identifying a body part of the person from the person image; The item receiving means displays the second item options while excluding items to be worn on parts of the person image that are not identified from the options, or displays the second item options while graying out the items to be worn on parts of the person image that are not identified. The virtual try-on system according to claim 1 or 2.

6. further comprising a body part identification means for identifying a body part of the person from the person image; The article receiving means outputs a warning when an article to be worn on a part not identified in the person image is selected by the user as the second article. The virtual try-on system according to claim 1 or 2.

7. The output means outputs an article image showing the specified second article, separately from the wearing image. The virtual try-on system according to claim 1 or 2.

8. the wearing mode receiving means receives a designation of a wearing mode of the second article based on a user's selection of article images showing different appearance shapes of the second article for each wearing mode from among a plurality of wearing modes; The output means displays a plurality of the article images side by side as options for wearing modes. The virtual try-on system according to claim 1 or 2.

9. The computer Acquire a person image of a person who will be performing the virtual try-on; Identifying a first item worn by the person based on the image of the person; Accept the designation of the second item to be virtually tried on; Accepting a designation of a wearing style of the second article; Based on the body of the person and the first item identified from the person image and the specified second item, a wearing image is output that shows the appearance of the second item in the specified wearing manner and makes the person appear to be wearing the second item. Virtual try-on method.

10. Acquire a person image of a person who will be performing the virtual try-on; Identifying a first item worn by the person based on the image of the person; Accept the designation of the second item to be virtually tried on; Accepting a designation of a wearing style of the second article; Based on the body of the person and the first item identified from the person image and the specified second item, a wearing image is output that shows the appearance of the second item in the specified wearing manner and makes the person appear to be wearing the second item. A program that causes a computer to perform a process.

Images