Information processing device, information processing method, and information processing program

Abstract

Provided are an information processing device, an information processing method, and an information processing program capable of encouraging a user to purchase clothing by making it easier to visualize the clothing when worn by the user. The information processing device: stores product information relating to clothing; stores model-wearing photographs, which are photographs of models wearing clothing related to the stored product information; stores model information including body type information of each model; stores user information including body type information of a user who is purchasing clothing; extracts a model having a similar body type to that of the user on the basis of the stored user information and the stored model information; provides the user with the model-wearing photograph relating to the extracted model; acquires a facial image of the user; generates a user-wearing photograph in which the face of the model in the provided model-wearing photograph has been changed to the user's face based on the acquired facial image; and provides the generated user-wearing photograph to the user.

Description

Information processing device, information processing method, and information processing program

[0001] The present invention relates to an information processing device, an information processing method, and an information processing program.

[0002] Conventionally, a wide variety of websites have been established for purchasing clothing. Users can browse coordinated photos of products and models wearing the clothing (sometimes abbreviated as "coordinate photos") on the website of their favorite brand or on sales sites that sell products from various brands, without having to visit a physical store. Users can imagine how the clothing they want to purchase will look on themselves by looking at the coordinated photos, and then purchase the product through e-commerce.

[0003] Since it is not possible to try on clothes online like in a physical store, it is difficult to know how the size will feel when worn by the customer. For this reason, a technology has been disclosed that can determine the fit of clothes online.

[0004] For example, Patent Document 1 describes a database in which measurements of each part of a product used in virtual fitting and measurements of each part of a purchaser's body are registered, and a technology that obtains measurements of the size of each part of a product and each part of a purchaser's body, and compares these to determine how well the product fits the purchaser.

[0005] JP 2017-134470 A

[0006] However, users cannot imagine what the clothes will look like when worn based solely on how well the clothes fit. Also, in coordinated photos of models wearing the clothes, the differences between the model's body shape and facial features and the user's own body shape and facial features can make it difficult for users to imagine what the clothes will look like when worn, which can lead to hesitation in purchasing the clothes.

[0007] The present invention has been made in consideration of the above circumstances, and one of its objectives is to provide an information processing device, an information processing method, and an information processing program that can encourage users to purchase clothing by making it easier for them to imagine what the clothing will look like when worn.

[0008] In order to solve the above problems, the information processing device of the embodiment includes: a product information storage unit that stores product information for clothing; a wearing photo storage unit that stores model wearing photos, which are photos of models wearing clothing related to the product information stored in the product information storage unit; a model information storage unit that stores model information including body type information of the model; a user information storage unit that stores user information including body type information of the user who purchases the clothing; a model extraction unit that extracts a model similar to the body type of the user based on the user information stored in the user information storage unit and the model information stored in the model information storage unit; a model wearing photo providing unit that provides a user with a model wearing photo related to the model extracted in the model extraction unit; a face image acquisition unit that acquires a face image of the user; a user wearing photo generation unit that generates a user wearing photo in which the face of the model in the model wearing photo provided by the model wearing photo provision unit is changed to the face of the user based on the face image acquired by the face image acquisition unit; and a user wearing photo provision unit that provides a user with the user wearing photo generated by the user wearing photo generation unit.

[0009] According to one embodiment of the present invention, product information for clothing is stored, a model-wearing photo is stored which is a photo of a model wearing clothing related to the stored product information, model information including body type information of the model is stored, user information including body type information of the user purchasing the clothing is stored, a model similar to the user's body type is extracted based on the stored user information and the stored model information, the model-wearing photo related to the extracted model is provided to the user, a facial image of the user is obtained, a user-wearing photo is generated in which the face of the model in the provided model-wearing photo is changed to the face of the user based on the obtained facial image, and the generated user-wearing photo is provided to the user, thereby making it easier for the user to imagine what the clothing will look like when worn, thereby promoting the purchase of the clothing.

[0010] 1 is a diagram showing an example of an overview of an information processing device in an embodiment; FIG. 2 is a diagram showing an example of a configuration of an information processing device in an embodiment; FIG. 3 is a diagram showing an example of model information in an embodiment; FIG. 4 is a diagram showing an example of user information in an embodiment; FIG. 5 is a flowchart showing an example of a first operation of an information processing device in an embodiment; FIG. 6 is a flowchart showing an example of a first operation of an information processing device in an embodiment; FIG. 7 is a diagram showing an example of generation of a photo of a user wearing the item in an embodiment; FIG. 8 is a flowchart showing an example of a second operation of an information processing device in an embodiment; FIG. 9 is a flowchart showing an example of a third operation of an information processing device in an embodiment; FIG. 10 is a block diagram showing an example of a hardware configuration of an information processing device in an embodiment.

[0011] An information processing device, an information processing method, and an information processing program according to an embodiment of the present invention will be described in detail below with reference to the drawings.

[0012] First, an overview of an information processing apparatus will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of an overview of an information processing apparatus according to an embodiment.

[0013] In FIG. 1 , an information processing device 1 is communicatively connected to a user terminal 2 used by a user who purchases clothing. The information processing device 1 is a website that sells clothing and provides product information about the clothing to the user terminal 2. Product information is information that is referenced when purchasing clothing, such as the size, material, price, color, or photo of the clothing. The product information may also include a photo of a model wearing the clothing (hereinafter referred to as a "model wearing photo"). In this embodiment, a model is a person or doll wearing the clothing. A model wearing photo may be, for example, a photo of a fashion model wearing the clothing taken by the product's manufacturer, a photo of a sales clerk wearing the clothing, or a photo of another user (such as a reader model in a fashion magazine) wearing the clothing. The model wearing photo may also be a coordinated combination of the clothing with other clothing (sometimes abbreviated as "coordinate"). By viewing the coordinated photo, users can imagine how the clothing they are purchasing will look or the situations in which they will be worn.

[0014] The information processing device 1 may acquire a facial image of the user from the user terminal 2. The facial image of the user includes a facial photograph of the user, an edited photograph obtained by editing the facial photograph of the user, or a portrait of the user. Based on the acquired facial image, the information processing device 1 generates a user-wearing photograph in which the face of the model in the model-wearing photograph is changed to the facial image of the user, and provides the generated photograph to the user terminal 2. Details of generating a user-wearing photograph will be described later.

[0015] The information processing device 1 may also acquire web images from the user terminal 2. Web images are images other than photos of models wearing the clothing provided by the information processing device 1 and include clothing. The information processing device 1 searches for clothing similar to the clothing included in the acquired web images from clothing sold on its own website and provides the user terminal 2 with photos of models wearing the searched similar clothing. Web images include, for example, photos posted to SNS (social networking service). Users who view photos posted on SNS may become interested in the clothing included in the posted photos. By searching for clothing similar to the clothing included in the posted photos, the information processing device 1 can link the user's interest to clothing sales.

[0016] The information processing device 1 may also be communicatively connected to a store terminal 3 installed in a physical store that sells clothing. The store terminal 3 is a device for providing a photo of a user wearing the clothing in the physical store. The store terminal 3 has a display, a camera, and a sensor arranged in the physical store that acquires terminal information of the user terminal 2. The sensor that acquires the terminal information of the user terminal 2 is, for example, a short-range wireless sensor that performs short-range wireless communication with the user terminal 2 to acquire the terminal information. The sensor is arranged, for example, near a product for which a photo of the user wearing the clothing is to be generated. When an owner of the user terminal 2 approaches the product in the physical store, the sensor reads the terminal information of the user terminal 2. The information processing device 1 acquires the terminal information read by the sensor via the store terminal 3 and identifies the user. The camera captures a facial image (still image or video) of the user approaching the product. The information processing device 1 acquires the captured facial image via the store terminal 3 and generates a photo of the user wearing the product. The information processing device 1 displays the generated photo of the user wearing the product on a display via the store terminal 3. This allows users to get an idea of ​​how the products will look when worn in store without having to try them on in a physical store.

[0017] Next, the configuration of the information processing device 1 will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device according to the embodiment.

[0018] 2 , an information processing device 1 is communicably connected to a user terminal 2 and a store terminal 3 via a network 9. The information processing device 1 has functional units, including a product information storage unit 11, a wearing photo storage unit 12, a model information storage unit 13, a user information storage unit 14, a model extraction unit 15, a model wearing photo provision unit 16, a face image acquisition unit 17, a user wearing photo generation unit 18, a user wearing photo provision unit 19, a user terminal detection unit 20, a user information search unit 21, a color change unit 22, an image acquisition unit 23, and a clothing search unit 24. The above functional units of the information processing device 1 in this embodiment will be described as functional modules realized by an information processing program (software) in this embodiment.

[0019] The product information storage unit 11 stores product information for clothing. As described above, product information is information that is referred to when purchasing clothing, such as the size, material, price, color, or photo of the clothing.

[0020] The wearing photo storage unit 12 stores model wearing photos, which are photos of models wearing the clothing related to the product information stored in the product information storage unit 11. The wearing photo storage unit 12 can store photos of models wearing the clothing of various body types. The wearing photo storage unit 12 can also store photos of one model wearing clothing of various sizes. The wearing photo storage unit 12 can also store photos of one model wearing clothing of various colors.

[0021] As described above, the product information may include a photo of a model wearing the product. However, for example, if the photo of a model wearing the product is a photo of a sales clerk at a retail store wearing the clothing, the photo of the model wearing the product may differ from store to store. Furthermore, the photo of the model wearing the product may differ depending on the region or country of the retail store. The wearing photo storage unit 12 stores the photos of the model wearing the product independently of the product information storage unit 11, thereby facilitating the management of the photos of the model wearing the product. Note that the photos of the model wearing the product stored in the wearing photo storage unit 12 are associated with model information, which will be described later, and identifying the photos of the model wearing the product allows a unique model to be identified.

[0022] The model information storage unit 13 stores model information including body type information of the model. The model information stored in the model information storage unit 13 will now be described with reference to Fig. 3. Fig. 3 is a diagram showing an example of model information in this embodiment.

[0023] In FIG. 3 , the model information 131 includes information such as a model number, height, bust, waist, hips, age, weight, size, and size number (hereinafter, the information on height, bust, waist, hips, age, weight, size, and size number will be referred to as "body type information"). The model information 131 stored in the model information storage unit 13 is information related to the model in the model wearing photo stored in the wearing photo storage unit 12. The model number is a number for identifying the model. For example, by associating the model number with the model wearing photo stored in the wearing photo storage unit 12, it becomes possible to associate the model wearing photo with the model information 131. The model's body type information is information that represents the model's body type. The size is a symbol such as XS, S, M, L, or XL. Since sizes may be displayed differently depending on the clothing manufacturer, the size information may also include manufacturer information. The size number is information related to the clothing size specified in JIS L4005 or the like.

[0024] Returning to Fig. 2, the user information storage unit 14 stores user information including body type information of the user. Here, the user information stored in the user information storage unit 14 will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of user information in the embodiment.

[0025] 4, user information 141 includes information such as membership number, name, address, telephone number, terminal information, purchase history, payment information, and point information. The user information 141 also includes information about the user's physique, such as height, bust, waist, hips, size, and number. The terminal information is information for identifying the user terminal 2 used by the user. A unique user can be identified based on the terminal information. The purchase history is information about purchase history on a website, and can be used, for example, for marketing that analyzes preferences for clothing purchases.

[0026] Returning to FIG. 2 , the model extraction unit 15 extracts a model similar to the user's body type based on the user information stored in the user information storage unit 14 and the model information stored in the model information storage unit 13. The model extraction unit 15 compares the user's body type information stored in the user information storage unit 14 with the body type information of the model stored in the model information storage unit 13 to extract a model with a body type similar to the user's body type. For example, if the product is a skirt, the model extraction unit 15 extracts a model with a waist similar to the user's waist based on the user information and waist information in the model information. The model extraction unit 15 may also extract a model based on age or weight information. If the model extraction unit 15 cannot extract a model with similar body type information, it may extract a model with the closest body type information.

[0027] The model wearing photo providing unit 16 provides the user with a photo of the model wearing the garment, which is related to the model extracted by the model extracting unit 15. That is, the model wearing photo providing unit 16 provides a photo of a model wearing the garment, which is of a model with a body type similar to that of the user, from among the multiple photos of models wearing the garment stored in the wearing photo storage unit 12. This allows the user to know the size of the garment when they try it on.

[0028] The model wearing photo providing unit 16 may provide a photo of a model wearing the item in a selected size based on the user's selection. For example, the impression of the item when worn may differ between when the user wears the item in a snug fit and when the user wears the item in a loose fit that is one size larger. By providing a photo of a model wearing the item in a selected size based on the user's selection, the model wearing photo providing unit 16 can provide the user with an impression of the item being worn in different sizes.

[0029] The facial image acquisition unit 17 acquires a facial image of the user. As described above, the facial image of the user is a facial photograph of the user, an edited photograph of the user's facial photograph, a portrait of the user, etc. The facial image acquisition unit 17 acquires, for example, a facial photograph taken with a camera of the user terminal 2, a facial image stored in the user terminal 2, a facial image shared from an application of the user terminal 2, or a facial photograph taken with a camera connected to the store terminal 3.

[0030] The user wearing photo generation unit 18 generates a user wearing photo by changing the face of the model in the model wearing photo provided by the model wearing photo provision unit 16 to the face of the user based on the face image acquired by the face image acquisition unit 17. The user's face based on the face image acquired by the face image acquisition unit 17 is the face image acquired by the face image acquisition unit 17 or a processed image obtained by processing the acquired face image. For example, if the orientation of the model's face in the model wearing photo is shifted from the front, the user wearing photo generation unit 18 corrects the orientation of the face in the acquired face image of the user to the orientation of the model's face, and generates a user wearing photo.

[0031] The user wearing photo generation unit 18 may generate a user wearing photo by changing the background of the model wearing photo. For example, the user wearing photo generation unit 18 generates a user wearing photo by combining a model with a background image selected by the user. The background image may be a prepared image provided to the user terminal 2, or the user may select an arbitrary image.

[0032] The user wearing photo generation unit 18 may also generate a user wearing photo in a still image / video compatible manner. Still image / video compatible refers to one of the following: A: generating a still user wearing photo when the model wearing photo is a still image; B: generating a video user wearing photo when the model wearing photo is a still image; C: generating a still user wearing photo when the model wearing photo is a video; or D: generating a video user wearing photo when the model wearing photo is a video. For example, in the case of B, a video combining multiple still images is generated. In the case of C, a still image obtained by cutting out a part of a video may be generated. The user wearing photo generation unit 18 may also generate a user wearing photo in a user face image modified to match the image quality, color tone, brightness, saturation, etc. of the model wearing photo.

[0033] The user wearing photo generation unit 18 may generate a user wearing photo in which the hairstyle of the model in the model wearing photo is further changed to the user's hairstyle based on the face image acquired by the face image acquisition unit 17. The user's hairstyle based on the face image, like the face image, is the user's hairstyle image acquired by the face image acquisition unit 17, or an edited image obtained by editing the acquired hairstyle image. By changing the hairstyle in the model wearing photo to the user's hairstyle, the user can more easily imagine what the user will look like wearing the clothing. Furthermore, the user wearing photo generation unit 18 may change the model's hairstyle in the model wearing photo to a pre-stored hairstyle. The impression of wearing clothing may change depending on the hairstyle. For example, by changing the hairstyle to match the outfit in the model wearing photo, the user can consider a hairstyle that suits the outfit.

[0034] The user wearing photo providing unit 19 provides the user with the photo of the user wearing the garment generated by the user wearing photo generating unit 18. The user wearing photo providing unit 19 can provide the user with the photo of the user wearing the garment, for example, by displaying the photo of the user wearing the garment on a display (not shown) of the user terminal 2 or a display (not shown) installed in a store connected to the store terminal 3. By providing the user with a photo of the user wearing the garment in which the face of the model in the photo of the model wearing the garment has been changed to the face of the user, the user can easily imagine what the garment will look like when worn. This motivates the user to purchase the garment and promotes the purchase of the garment.

[0035] The user terminal detection unit 20 detects a user terminal 2 in a physical store that sells clothing. The user terminal detection unit 20 can detect the user terminal 2, for example, based on terminal information provided from the user terminal 2 in an application installed on the user terminal 2. The terminal information is acquired, for example, by a sensor installed in the physical store. The user terminal detection unit 20 can detect the user terminal 2 by acquiring the terminal information acquired by the sensor via the store terminal 3.

[0036] The user information search unit 21 searches the user information stored in the user information storage unit 14 based on the terminal information of the user terminal 2 detected by the user terminal detection unit 20, and identifies the user who owns the user terminal 2. This enables the user information search unit 21 to link the user who owns the user terminal 2 found in the store with the user's body type information. Note that, although the present embodiment has described a case in which one user terminal 2 is stored in the user information storage unit 14 in association with one user, the user information storage unit 14 may also store multiple user terminals 2 in association with one user.

[0037] The color change unit 22 changes the color of the clothing in the model wearing photos stored in the wearing photo storage unit 12. Clothing may be sold in multiple colors. Preparing photos of models wearing the clothing for all colors increases the amount of work required to take the photos and the storage capacity required to store the photos. By changing the color of the clothing in the model wearing photos stored in the wearing photo storage unit 12, the color change unit 22 can generate photos of models wearing the clothing in multiple colors from a single photo, thereby reducing the amount of work required for photography and the storage capacity required. The color change unit 22 may also change the pattern of the clothing, marks attached to the clothing, or the texture of the clothing material.

[0038] The image acquisition unit 23 acquires an image including clothing. As described above, an image including clothing is, for example, a photo posted on a social networking site. The image including clothing may be a video distributed on a video distribution site. The image including clothing may also be a photographed image taken by a user.

[0039] The clothing search unit 24 may search the product information stored in the product information storage unit 11 for clothing similar to the clothing included in the image acquired by the image acquisition unit 23. The model-wearing-photo provision unit 16 can provide photos of models wearing the similar clothing found by the clothing search unit 24. This makes it possible to promote sales of clothing similar to the clothing included in the image.

[0040] Note that the above-described functional units of the information processing device 1 are merely examples of the functional units of the information processing device 1 and do not limit the functions of the information processing device 1. For example, the information processing device 1 does not need to have all of the above-described functional units, and may have only some of the functional units. Furthermore, the information processing device 1 may have functions other than those described above. For example, the information processing device 1 may have an input function for inputting information, or an output function for notifying the operating status of the device using an LED lamp or the like.

[0041] As described above, the functional units of the information processing device 1 are implemented by software. However, at least one of the functional units may be implemented by hardware.

[0042] Furthermore, any of the above functional units may be implemented by dividing one functional unit into multiple functional units. Furthermore, any two or more of the above functional units may be implemented by integrating them into one functional unit. Furthermore, the above description expresses the functions of the information processing device 1 using functional blocks, and does not indicate, for example, that each functional unit is configured by a separate program file or the like.

[0043] Furthermore, the information processing device 1 may be a device realized by a single housing, or a system realized by multiple devices connected via a network, etc. For example, the information processing device 1 may realize some or all of its functions by a virtual device, such as a cloud service provided by a cloud computing system. That is, the information processing device 1 may realize at least one or more of the above-mentioned functional units in another device. Furthermore, the information processing device 1 may be a general-purpose computer, such as a desktop PC, or a dedicated device with limited functions.

[0044] Next, the operation of the information processing device 1 will be described with reference to Figures 5 and 6. Figures 5 and 6 are flowcharts showing an example of the operation of the information processing device in the embodiment. Note that, although the process in the flowcharts described below will be described as being executed by the information processing device 1, each process is executed by each functional unit of the information processing device 1 described above.

[0045] 5, the information processing device 1 provides the product status to the user terminal 2 (step S11). After executing the process of step S11, the information processing device 1 determines whether or not a product has been selected by the user (step S12). If it determines that a product has not been selected (step S12: NO), the information processing device 1 repeats the process of step S12 and waits for a product to be selected.

[0046] On the other hand, if it is determined that a product has been selected (step S12: YES), the information processing device 1 determines whether or not to display a coordinated outfit photo based on a user operation (step S13). In this embodiment, a coordinated outfit photo is a photo of the user wearing clothing for which a model has not been extracted from the user's body type information, and is, for example, a photo displayed to provide an overview of the clothing. If it is determined not to display a coordinated outfit photo (step S13: NO), the information processing device 1 repeats the process of step S13 and waits for the user to perform an operation to display the coordinated outfit photo.

[0047] On the other hand, if it is determined that the outfit photo should be displayed (step S13: YES), the information processing device 1 provides the outfit photo to the user (step S14). After executing the process of step S14, the information processing device 1 determines whether or not to conduct a virtual try-on (step S15). In this embodiment, the virtual try-on refers to generating and displaying a photo of the user wearing the above-mentioned item. If it is determined that the virtual try-on will not be conducted (step S15: NO), the information processing device 1 repeats the process of step S15 and waits for the virtual try-on to be conducted.

[0048] On the other hand, if it is determined that a virtual try-on will be performed (step S15: YES), the information processing device 1 executes a model extraction process (step S16). After executing the process of step S16, the information processing device 1 provides a photo of a model wearing the garment based on the user's body type information (step S17). After executing the process of step S17, the information processing device 1 determines whether to execute a face change (step S18). In this embodiment, a face change refers to generating a photo of the user wearing the garment. If it is determined that a face change will not be executed (step S18: NO), the information processing device 1 repeats the process of step S18 and waits for the face change to be executed.

[0049] On the other hand, if it is determined that a face change is to be performed (step S18: YES), the information processing device 1 provides the user with a facial photo selection screen (step S19). The facial photo selection screen is a screen that is displayed on the user terminal 2 and allows the user to select a facial image of the user. For example, the user can select a facial photo taken with the camera of the user terminal 2 or an image file stored in the user terminal 2. After executing the processing of step S19, the information processing device 1 determines whether a facial image has been acquired (step S20). If it is determined that a facial image has not been acquired (step S20: NO), the information processing device 1 repeats the processing of step S20 and waits for a facial image to be acquired.

[0050] On the other hand, if it is determined that a face image has been acquired (step S20: YES), the information processing device 1 generates a photo of the user wearing the device (step S21). Here, the process of step S21 will be described with reference to FIG.

[0051] 6, the information processing device 1 determines whether the user has performed an operation to change the hairstyle (step S211). If it is determined that the user has performed an operation to change the hairstyle (step S211: YES), the information processing device 1 extracts the user's hairstyle included in the face image (step S212). After performing the process of step S212, the information processing device 1 performs a process of combining the face and hairstyle in generating a photo of the user wearing the garment (step S213).

[0052] On the other hand, if it is determined that the operation to change the hairstyle is not performed (step S211: NO), the information processing device 1 executes a process of compositing a face in generating a photo of the user wearing the device (step S214). After executing the process of step S213 or the process of step S214, the information processing device 1 ends the process of step S21.

[0053] Returning to FIG. 5, after executing the process of step S21, the information processing device 1 provides the user with a photograph of the user wearing the garment (step S22), and ends the operation shown in the flowchart.

[0054] Next, generation of a photo of a user wearing the product will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of generation of a photo of a user wearing the product in the embodiment.

[0055] 7, model wearing photo 1001 is a photo selected by a user from multiple photos of models wearing the garment. Facial image 1002 is a facial image of the user selected by the user. User wearing photo 1003 is a composite image in which the face of the model in model wearing photo 1001 is replaced with the face of the user in facial image 1002. In user wearing photo 1003, the user's face in facial image 1002 is tilted relative to the tilt of the model's face in model wearing photo 1001. By combining their own face with a photo of a model wearing the garment who has a similar body type to their own, the user can more easily imagine what the garment will look like when worn by the user.

[0056] Next, a second operation of the information processing device will be described with reference to Fig. 8. Fig. 8 is a flowchart showing an example of the second operation of the information processing device in the embodiment.

[0057] 8, the information processing device 1 determines whether or not the user terminal 2 has been detected in the physical store (step S21). If it is determined that the user terminal 2 has not been detected (step S21: NO), the information processing device 1 repeats the process of step S21 and waits for the user terminal 2 to be detected.

[0058] On the other hand, if it is determined that a user terminal 2 has been detected (step S21: YES), the information processing device 1 identifies the user of the user terminal 2 from the terminal information of the detected user terminal 2 and extracts a model with a similar body type based on the user's body type information (step S22). After executing the process of step S22, the information processing device 1 determines whether a facial image captured by a camera installed in a physical store has been acquired (step S23). If it is determined that a facial image has not been acquired (step S23: NO), the information processing device 1 repeats the process of step S23 and waits for a facial image to be acquired.

[0059] On the other hand, if it is determined that a face image has been acquired (step S23: YES), the information processing device 1 generates a photo of the user wearing the product (step S24). After executing the process of step S24, the information processing device 1 provides the generated photo of the user wearing the product to a display installed in a physical store (step S25), and ends the operation shown in the flowchart.

[0060] Next, a third operation of the information processing device 1 will be described with reference to Fig. 9. Fig. 9 is a flowchart showing an example of the third operation of the information processing device in the embodiment.

[0061] 9, the information processing device 1 determines whether a web image has been acquired from a user (step S31). If it is determined that a web image has not been acquired (step S31: NO), the information processing device 1 repeats the process of step S31 and waits for the web image to be acquired.

[0062] On the other hand, if it is determined that a web image has been acquired (step S31: YES), the information processing device 1 searches the product information for similar clothing to the clothing included in the web image (step S32). After executing the process of step S32, the information processing device 1 provides the user with product information on the searched similar clothing (step S33). After executing the process of step S33, the information processing device 1 generates a photo of the user wearing the clothing, as described in Figures 5 and 6 (step S34), and ends the operation shown in the flowchart.

[0063] Note that the operations shown in the above flowcharts are merely examples of the operations of the information processing device 1 and do not limit the order of processing. For example, specific processing may be executed in an order or at a timing different from the processing order in the flowcharts.

[0064] Next, the hardware configuration of the information processing device 1 will be described with reference to Fig. 10. Fig. 10 is a block diagram showing an example of the hardware configuration of the information processing device according to the embodiment.

[0065] The information processing device 1 includes a CPU (Central Processing Unit) 101 , a RAM (Random Access Memory) 102 , a ROM (Read Only Memory) 103 , an I / O device 104 , and a communication I / F (Interface) 105 .

[0066] The CPU 101 controls the information processing device 1 by executing an information processing program stored in the RAM 102 or the ROM 103. The information processing program is obtained, for example, from a recording medium on which the program is recorded or from a program distribution server via a network, and is installed in the ROM 103, and is read out and executed by the CPU 101.

[0067] The I / O device 104 has an operation input function and a display function (operation display function). The I / O device 104 is, for example, a touch panel. The touch panel allows the user of the information processing device 1 to input operations using a fingertip, a touch pen, or the like. In this embodiment, the I / O device 104 is described as using a touch panel with an operation display function. However, the I / O device 104 may also have a separate display device with a display function and an operation input device with an operation input function. In this case, the display screen of the touch panel can be implemented as the display screen of the display device, and operation of the touch panel can be implemented as operation of the operation input device. The I / O device 104 may also be implemented in various forms, such as a head-mounted, eyeglass-type, or wristwatch-type display.

[0068] The communication I / F 105 is an I / F for communication. The communication I / F 105 executes short-range wireless communication such as wireless LAN, wired LAN, infrared, etc. Although the figure shows only the communication I / F 105 as the communication I / F, the information processing device 1 may have communication I / Fs for multiple communication methods.

[0069] Although the embodiments of the present invention have been described above with reference to the drawings, the specific configuration is not limited to this embodiment, and various modifications are also included within the scope of the gist of the present invention. In addition to the description of the above-mentioned embodiments, the embodiments include the following aspects.

[0070] The face image acquisition unit 17 can acquire a whole-body image of the user, including a face image of the user.

[0071] The information processing device 1 acquires a full-body image of the user provided by the user using the facial image acquisition unit 17. The model extraction unit 15 uses artificial intelligence (AI, or other technology that realizes human intelligence on a computer) to extract a model based on the full-body image of the user provided by the user. The model wearing photo provision unit 16 can provide the model wearing photo extracted by the model extraction unit 15 to the user terminal 2. AI extracts features from the user's full-body image and suggests coordination, which can lead to encouraging users to purchase clothing. In addition to the user's full-body image provided by the user, the model extraction unit 15 can also extract a model based on a specific event selected by the user (such as a wedding or interview), weather, skin color, race, body type, skeletal type, personal color, specific age group, region, or gender. When providing the model wearing photos to the user terminal 2, the information processing device 1 can provide the model wearing photos in an order determined to suit the user. Furthermore, when providing a photo of a model wearing the product to the user terminal 2, the information processing device 1 can also provide text information generated by AI in addition to the photo of the model wearing the product.

[0072] Furthermore, the information processing device 1 can extract a model using the model extraction unit 15 based on the user's past purchase history stored in the user information storage unit 14. The model wearing photo providing unit 16 may provide the model wearing photo extracted by the model extraction unit 15 to the user terminal 2 and the store terminal 3. The model extraction unit 15 can also extract a model by referring to product discount information in the process of extracting a model. The model extraction unit 15 can also use the user wearing photo stored in the user information storage unit 14 as a reference for model extraction.

[0073] Furthermore, the photos of the user wearing the clothing and the user's try-on history stored in the user information storage unit 14 can be used for purposes such as the user publishing the photos on social media and receiving evaluations from other users. For example, a user can post a photo of themselves wearing the clothing on social media and receive feedback from their followers, thereby receiving feedback on how the clothing looks. In addition, for example, a user may like a photo of another user wearing the clothing. In such cases, the information processing device 1 may sell the right to purchase the clothing on the user terminal 2 or the right to use the clothing as a profile picture.

[0074] The information processing device 1 can search for similar facial images based on the facial image of the user acquired by the facial image acquisition unit 17 and comparing it with facial images of other users stored in the user information storage unit 14. Based on the search results, the purchase histories of other users with similar facial images can be acquired and displayed on the user terminal 2. For example, the information processing device 1 can score the recommendation level of a product based on purchase and try-on data of users with similar facial features and present the score to the user, thereby providing information to encourage purchases. Furthermore, by making it possible to refer to photos of users with similar faces wearing the product, it is possible to provide reference information when the user is considering a purchase. Note that when providing photos of users with similar faces wearing the product to the user terminal 2, the faces may be illustrated.

[0075] The information processing device 1 can analyze the user's facial image acquired by the facial image acquisition unit 17 and detect the shape of the face. Based on the detection results, it is possible to suggest necklines and collar styles suited to the user's facial shape and provide them to the user terminal 2. This suggestion function allows the user to select the most suitable clothing based on their facial features.

[0076] The information processing device 1 can utilize the photo of the user wearing the clothing generated by the user wearing photo generation unit 18. For example, in a physical store, the photo of the user wearing the clothing can be displayed on a display in the storefront and provided as reference information to other customers as "examples of actual wear of this clothing," thereby visually conveying the appeal of the product.

[0077] The information processing device 1 provides the user terminal 2 with re-advertisements, sale information, and new product information for products that the user is interested in, based on the photos of the user wearing the product acquired by the user wearing photo providing unit 19 and the user information storage unit 14. The information processing device 1 also has a function for storing online try-on history in the user information storage unit 14, which can be used in physical stores. Specifically, by acquiring a facial photo of the customer at the store terminal 3 and linking it with online membership information, store employees can understand the user's preferences and purchasing trends. This allows employees to propose products and provide customer service tailored to the customer, encouraging purchases.

[0078] The wearing photo memory unit 12 stores model wearing photos, which are photos of models wearing clothing related to the product information stored in the product information memory unit 11. The wearing photo memory unit 12 can store wearing photos of models with various body types. The wearing photo memory unit 12 can also store wearing photos of a single model wearing clothing of various sizes. The wearing photo memory unit 12 can also store wearing photos of a single model wearing clothing of various colors. The wearing photo memory unit 12 can also store images of a model enjoying sightseeing in traditional clothing, wearing a sports uniform, in a luxury hotel suite, in first class on an airplane, and taking photos with athletes at a stadium where professional sports are held. By using these model wearing photos, the information processing device 1 can provide the effect of promoting purchases, increasing motivation to travel, and increasing motivation to watch sports in person.

[0079] The model wearing photo providing unit 16 may provide a photo of the model wearing the selected garment with some of the garment specifications changed based on the user's selection. For example, the impression of the garment when worn may differ depending on whether the user wears the garment without rolling up the sleeves or with the sleeves rolled up. By providing a photo of the model wearing the selected garment with some of the garment specifications changed based on the user's selection, the model wearing photo providing unit 16 can provide the user with an impression of how the sleeves of the garment will look when worn.

[0080] The information processing device 1 may also be communicatively connected to a store terminal 3 installed in a physical store that sells clothing. The store terminal 3 is a device for providing a photo of a user wearing the clothing in the physical store. The store terminal 3 has a display, a camera, and a sensor arranged in the physical store that acquires terminal information of the user terminal 2. The sensor that acquires the terminal information of the user terminal 2 is, for example, a short-range wireless sensor that performs short-range wireless communication with the user terminal 2 to acquire the terminal information. The sensor is arranged, for example, near a product for which a photo of the user wearing the clothing is to be generated. When an owner of the user terminal 2 approaches the product in the physical store, the sensor reads the terminal information of the user terminal 2. The information processing device 1 acquires the terminal information read by the sensor via the store terminal 3 and identifies the user. The camera captures a facial image (still image or video) of the user approaching the product. The information processing device 1 acquires the captured facial image via the store terminal 3 and generates a photo of the user wearing the product. The information processing device 1 displays the generated photo of the user wearing the product on a display via the store terminal 3. This allows the user to get an idea of ​​how the product will look in the store without having to try it on in a physical store. Furthermore, the information processing device 1 can recognize products even if they are out of stock at the store by reading the product's barcode with the camera of the store terminal 3. By reading the product's barcode and tag with the camera of the store terminal 3, the information processing device 1 can present the product's stock information and size information stored in the product information storage unit 11, and if the product is available at another store, how to order it, to the user terminal 2.

[0081] The user information storage unit 14 stores user information 141 including information on the user's body type and a photo of the user wearing the item.

[0082] The user wearing photo generation unit 18 generates a user wearing photo in which the face of the model in the model wearing photo provided by the model wearing photo providing unit 16 is changed to the face of the user based on the face image acquired by the face image acquisition unit 17. The user's face based on the face image acquired by the face image acquisition unit 17 is the face image acquired by the face image acquisition unit 17 or an edited image obtained by editing the acquired face image. For example, if the orientation of the model's face in the model wearing photo is shifted from the front, the user wearing photo generation unit 18 corrects the orientation of the face in the acquired user face image to the orientation of the model's face, thereby generating a user wearing photo. The user wearing photo generation unit 18 may change the facial expression of the user's facial image using AI. For example, the information processing device 1 can provide the user with an image of what it would look like if the facial expression was changed to a smile.

[0083] The user wearing photo generation unit 18 may generate a user wearing photo by changing the background of the model wearing photo. For example, the user wearing photo generation unit 18 generates a user wearing photo by combining a model with a background image selected by the user. The background image may be a prepared image provided to the user terminal 2, or the user may select an arbitrary image. Specifically, the background image provided may be an image according to the weather or season.

[0084] The user wearing photo generation unit 18 may generate a user wearing photo in which the hairstyle and hair color of the model in the model wearing photo are further changed to the hairstyle and hair color of the user based on the facial image acquired by the facial image acquisition unit 17. The user's hairstyle and hair color based on the facial image, like the facial image, are the user's hairstyle and hair color image acquired by the facial image acquisition unit 17, or an edited image obtained by editing the acquired hairstyle and hair color image. By changing the hairstyle and hair color in the model wearing photo to the user's hairstyle and hair color, the user can more easily imagine what the user will look like wearing the clothing. Furthermore, the user wearing photo generation unit 18 may change the hairstyle and hair color of the model in the model wearing photo to a hairstyle and hair color stored in advance. The impression of wearing the clothing may change depending on the hairstyle and hair color. For example, by changing the hairstyle and hair color to match the outfit in the model wearing photo, the user can consider a hairstyle and hair color that suits the outfit. The user wearing photo generating unit 18 may add items to be worn on the user's face, specifically items such as glasses and hats, to the user's face image at the user's selection.

[0085] The user wearing photo generation unit 18 may also generate a photo of the user wearing the product in a still image / video compatible manner. Still image / video compatibility refers to one of the following: A: generating a still image of the user wearing the product when the photo of the model wearing the product is a still image; B: generating a video of the user wearing the product when the photo of the model wearing the product is a still image; C: generating a still image of the user wearing the product when the photo of the model wearing the product is a video; or D: generating a video of the user wearing the product when the photo of the model wearing the product is a video. For example, in the case of B, a video combining multiple still images is generated. In the case of C, a still image obtained by cutting out a portion of the video may be generated. In the case of C, when the photo of the model wearing the product is a video, the video may be a video of the model walking. The user wearing photo generation unit 18 may also generate a photo of the user wearing the product in a facial image or a full-body image of the user that has been modified to match the image quality, color tone, brightness, saturation, etc. of the photo of the model wearing the product.

[0086] REFERENCE SIGNS LIST 1 Information processing device 2 User terminal 3 Store terminal 9 Network 11 Product information storage unit 12 Wearing photo storage unit 13 Model information storage unit 14 User information collection unit 15 Model extraction unit 16 Model wearing photo provision unit 17 Face image acquisition unit 18 User wearing photo generation unit 19 User wearing photo provision unit 20 User terminal detection unit 21 User information search unit 22 Color change unit 23 Image acquisition unit 24 Clothing search unit 101 CPU 102 RAM 103 ROM 104 Touch panel 105 Communication I / F

Claims

a model information storage unit that stores product information for clothing; a model-wearing-photo storage unit that stores model-wearing photos, which are photos of models wearing clothing related to the product information stored in the product information storage unit; a model information storage unit that stores model information including body type information of the models; a user information storage unit that stores user information including body type information of users who purchase the clothing; a model extraction unit that extracts the models similar to the body type of the users based on the user information stored in the user information storage unit and the model information stored in the model information storage unit; a model-wearing-photo provision unit that provides the users with the model-wearing-photos related to the models extracted by the model extraction unit; a face image acquisition unit that acquires a face image of the users; a user-wearing-photo generation unit that generates user-wearing-photos by changing the face of the model in the model-wearing-photos provided by the model-wearing-photo provision unit to the face of the user based on the face image acquired by the face image acquisition unit; and a user-wearing-photo provision unit that provides the users with the user-wearing-photos generated by the user-wearing-photo generation unit.

2. The information processing device according to claim 1, wherein the user-wearing photo generation unit generates a user-wearing photo in which the hairstyle of the model in the model-wearing photo is further changed to the hairstyle of the user based on the facial image acquired by the facial image acquisition unit.

3. An information processing device as described in claim 1 or 2, wherein the wearing photo memory unit stores photos of models wearing the clothing in multiple sizes, and the model wearing photo providing unit provides the photos of models wearing the clothing in the selected size based on the user's selection.

4. The information processing device according to claim 1, wherein the facial image acquisition unit acquires the facial image captured by a camera of a user terminal operated by a user.

5. An information processing device as described in claim 1, further comprising: a user terminal detection unit that detects a user terminal in a store selling clothing; and a user information search unit that searches for user information stored in the user information storage unit based on terminal information of the user terminal detected by the user terminal detection unit, wherein when the user terminal detection unit detects a user terminal, the facial image acquisition unit acquires the facial image taken by a camera installed in the store, the model extraction unit extracts the model based on the user information searched by the user information search unit, and the user-wearing-photo provision unit provides a photo of the user wearing the clothing to a display device installed in the store.

6. An information processing device as described in claim 1, further comprising a color change unit that changes the color of the clothing in the clothing photos stored in the clothing photo memory unit, and the model wearing photo providing unit provides the model wearing photo of the clothing whose color has been changed by the color change unit.

7. An information processing device as described in claim 1, further comprising: an image acquisition unit that acquires images including clothing; and a clothing search unit that searches product information stored in the product information storage unit for similar clothing that is similar to the clothing included in the images acquired by the image acquisition unit, wherein the model-wearing photo providing unit provides the model-wearing photo of the similar clothing searched for by the clothing search unit.

8. An information processing method comprising: a product information storing step of storing product information for clothing; a model wearing photo storing step of storing a photo of a model wearing the clothing related to the product information stored in the product information storing step; a model information storing step of storing model information including body type information of the model; a user information storing step of storing user information including body type information of a user who purchases the clothing; a model extraction step of extracting a model similar to the body type of the user based on the user information stored in the user information storing step and the model information stored in the model information storing step; a model wearing photo providing step of providing a user with the model wearing photo related to the model extracted in the model extraction step; a face image acquiring step of acquiring a face image of the user; a user wearing photo generating step of generating a user wearing photo in which the face of the model in the model wearing photo provided in the model wearing photo providing step is changed to the face of the user based on the face image acquired in the face image acquiring step; and a user wearing photo providing step of providing the user with the user wearing photo generated in the user wearing photo generating step.

9. An information processing program for causing a computer to implement the following processes: a product information storage process for storing product information about clothing; a model-wearing photo storage process for storing a model-wearing photo, which is a photo of a model wearing clothing related to the product information stored in the product information storage process; a model information storage process for storing model information including body type information of the model; a user information storage process for storing user information including body type information of a user who purchases the clothing; a model extraction process for extracting a model similar to the body type of the user based on the user information stored in the user information storage process and the model information stored in the model information storage process; a model-wearing photo provision process for providing a user with the model-wearing photo related to the model extracted in the model extraction process; a face image acquisition process for acquiring a facial image of the user; a user-wearing photo generation process for generating a user-wearing photo in which the face of the model in the model-wearing photo provided in the model-wearing photo provision process is changed to the face of the user based on the facial image acquired in the face image acquisition process; and a user-wearing photo provision process for providing the user with the user-wearing photo generated in the user-wearing photo generation process.

Images