Generation system, and generation program

Abstract

To provide a generation system and a generation program capable of allowing a user to confirm a state in which a styling target is applied. 【Solution means】 A server device 2 that generates a styling display image for displaying a user who is an application target in a state where clothing, which is a styling target, is applied. The server device 2 includes an acquisition unit that acquires body-related information indicating the body of the application target, a first generation unit that generates an application target display image for displaying the appearance of the application target based on the body-related information acquired by the acquisition unit, and a second generation unit that generates a styling display image based on the application target display image generated by the first generation unit and styling target-related information indicating the clothing that is the styling target. The first generation unit generates an application target display image based on the skeleton point information, which is the reference point information specified by the specifying unit, and the reference application target display image.

Description

Technical Field

[0001] The present invention relates to a generation system and a generation program.

Background Art

[0002] Conventionally, techniques for performing various processes on an image of a user trying on clothes have been known (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] By the way, for example, when purchasing a styling object such as clothes via the Internet, there has been a demand for a technique for allowing a user to confirm the state in which the styling object is applied.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a generation system and a generation program capable of allowing a user to confirm the state in which a styling object is applied.

Means for Solving the Problems

[0006] In order to solve the above-described problems and achieve the object, the generation system according to claim 1 is a generation system that generates a styling display image for displaying an applicable person in a state where a styling object is applied, and includes an acquisition unit that acquires body-related information indicating the body of the applicable person, a first generation unit that generates an applicable person display image for displaying the appearance of the applicable person based on the body-related information acquired by the acquisition unit, and a second generation unit that generates the styling display image based on the applicable person display image generated by the first generation unit and styling object-related information indicating the styling object. Based on the body-related information acquired by the acquisition means, a specifying means for specifying reference point information indicating the positions of a plurality of reference points serving as a reference for shaping the appearance of the body of the application target person, with reference to a reference application target person display image that is an image displaying the appearance of the reference application target person, is provided. The first generation means generates the application target person display image based on the reference point information specified by the specifying means and the reference application target person display image.

[0007] The generation system according to claim 2 is the generation system according to claim 1, wherein The reference point information is coordinates in a predetermined coordinate system in which the direction of the height of the reference application target person in the reference application target person display image is the first direction, and the direction orthogonal to the first direction is the second direction. The specifying means is based on a first ratio that is the ratio of the size of the height of the reference application target person to the number of pixels in the first direction corresponding to the height of the reference application target person in the reference application target person display image (the number of height pixels), a first calculation result based on the size of the reference application target person in a predetermined part in the second direction, a second ratio that is the ratio of the size of the height of the application target person to the number of height pixels, and a second calculation result based on the size of the application target person in the predetermined part in the second direction, and specifies the coordinate in the second direction in the reference point information.

[0008] The generation system according to claim 3 is the generation system according to claim 1 In the generation system described above, the specifying means performs a first specifying process of specifying the size of a predetermined part of the body of the applicable person based on the body-related information acquired by the acquisition means, and a second specifying process of specifying the reference point information based on the body-related information acquired by the acquisition means and the size of the predetermined part specified in the first specifying process. The second generation means generates the styling display image based on the size of the predetermined part of the body of the applicable person specified in the first specifying process, the applicable person display image generated by the first generation means, and the styling object-related information.

[0009] The generation system according to claim 4 is A generation system for generating a styling display image that displays an application target person in a state where a styling object is applied, including: an acquisition means for acquiring body-related information indicating the body of the application target person; a first generation means for generating an application target person display image that displays the appearance of the application target person based on the body-related information acquired by the acquisition means; a second generation means for generating the styling display image based on the application target person display image generated by the first generation means and styling object-related information indicating the styling object; and a specifying means for specifying reference point information indicating the positions of a plurality of reference points serving as a reference for shaping the appearance of the body of the application target person, with reference to a reference application target person display image that is an image displaying the appearance of the reference application target person, based on the body-related information acquired by the acquisition means. The first generation means generates the application target person display image based on the reference point information specified by the specifying means and the reference application target person display image. The specific means further specifies the obesity degree information of the applicable person based on the body-related information acquired by the acquisition means. The first generation means performs a first process on the first generation means side, which generates a first applicable person display image for displaying the appearance of the applicable person generated based on the whole body of the reference applicable person, based on the reference point information specified by the specific means, the obesity degree information specified by the specific means, and the reference applicable person display image. The first generation means also performs a second process on the first generation means side, which generates a second applicable person display image for displaying the appearance of the applicable person including the face corresponding to the face information by editing only the face of the applicable person shown in the first applicable person display image generated in the first process on the first generation means side, based on the face information showing the face part of the applicable person and input into the generation system. The second generation means generates the styling display image based on the second applicable person display image generated by the first generation means and the styling object-related information.

[0010] The generation system according to claim 5 is the generation system according to claim 1 Any one of 4 wherein the acquisition means further acquires the styling object-related information indicating the styling object selected by the user of the generation system. The styling object-related information includes at least a styling object display image for displaying the styling object and styling object size information indicating the size of the styling object. The second generation means generates the styling display image based on the applicable person display image generated by the first generation means, the styling object display image acquired by the acquisition means, and the styling object size information acquired by the acquisition means.

[0011] The generation program according to claim 6 is a generation program that generates a styling display image for displaying an applicable person in a state where a styling object is applied, and causes a computer to perform an acquisition means for acquiring body-related information indicating the body of the applicable person, and based on the body-related information acquired by the acquisition means, a first generation means for generating an applicable person display image for displaying the appearance of the applicable person, and a second generation means for generating the styling display image based on the applicable person display image generated by the first generation means and styling object-related information indicating the styling object. Based on the body-related information acquired by the acquisition means, as identification means for identifying reference point information indicating the positions of a plurality of reference points serving as a reference for shaping the appearance of the body of the application target person, with reference to a reference application target person display image which is an image displaying the appearance of the reference application target person, the first generation means generates the application target person display image based on the reference point information identified by the identification means and the reference application target person display image. The generation program according to claim 7 is a generation program that generates a styling display image for displaying an applicable person in a state where a styling object is applied. The computer is caused to function as an acquisition means for acquiring body-related information indicating the body of the applicable person, a first generation means for generating an applicable person display image for displaying the appearance of the applicable person based on the body-related information acquired by the acquisition means, a second generation means for generating the styling display image based on the applicable person display image generated by the first generation means and styling object-related information indicating the styling object, and a specifying means for specifying reference point information indicating the positions of a plurality of reference points that serve as a reference for shaping the appearance of the body of the applicable person, with reference to a reference applicable person display image for displaying the appearance of a reference applicable person based on the body-related information acquired by the acquisition means. The first generation means generates the applicable person display image based on the reference point information specified by the specifying means and the reference applicable person display image. The specifying means further specifies obesity information of the applicable person based on the body-related information acquired by the acquisition means. The first generation means performs a first process on the first generation means side, which is to generate a first applicable person display image for displaying the appearance of the applicable person generated with reference to the entire body of the reference applicable person, based on the reference point information specified by the specifying means, the obesity information specified by the specifying means, and the reference applicable person display image. The first generation means performs a second process on the first generation means side, which is to generate a second applicable person display image for displaying the appearance of the applicable person including the face corresponding to the face information by editing only the face of the applicable person shown in the first applicable person display image generated in the first process on the first generation means side, based on face information indicating the face of the applicable person and input to the computer. The second generation means generates the styling display image based on the second applicable person display image generated by the first generation means and the styling object-related information.

Advantages of the Invention

[0012] According to the generation system according to claim 1 and the generation program according to claim 6, by generating a styling display image, for example, it becomes possible to confirm a state where a styling object is applied. Also, according to the generation system described in claim 1 and the generation program described in claim 6, by using reference point information indicating the positions of a plurality of reference points serving as a standard for shaping the appearance of the body of the person to be applied, for example, a display image of the person to be applied can be appropriately generated, and it becomes possible to appropriately confirm the state in which the styling object is applied.

[0014] According to the generation system according to claim 3, by generating a styling display image based on the size of a predetermined part of the body of the applicable person, for example, it is possible to appropriately generate an applicable person display image in consideration of the size of a predetermined part of the body of the applicable person, and it becomes possible to appropriately confirm a state where a styling object is applied.

[0015] The generation system according to claim 4 And according to the generation program described in claim 7, by generating a styling display image, for example, it becomes possible to confirm the state in which the styling object is applied. Also, according to the generation system described in claim 4 and the generation program described in claim 7, by using reference point information indicating the positions of a plurality of reference points serving as a standard for shaping the appearance of the body of the person to be applied, for example, a display image of the person to be applied can be appropriately generated, and it becomes possible to appropriately confirm the state in which the styling object is applied. Also, according to the generation system described in claim 4 and the generation program described in claim 7, By performing a second process on the first generation means side for generating a second applicable person display image based on the first applicable person display image generated by the first process on the first generation means side, for example, it is possible to appropriately generate the second applicable person display image, and it becomes possible to appropriately confirm a state where a styling object is applied.

[0016] According to the generation system described in claim 5, by generating a styling display image based on the styling object display image and the styling object size information, for example, it becomes possible to appropriately confirm the state in which the styling object is applied.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Embodiments for Carrying Out the Invention

[0018] Hereinafter, embodiments of the generation system and generation program according to the present invention will be described in detail with reference to the drawings. However, the present invention is not limited by the embodiments. Here, after explaining the basic concept, specific embodiments will be described.

[0019] (Basic Concept) First, the basic concept will be explained. The generation system according to the present invention is a system that generates a styling display image for displaying an application target person in a state where a styling target object is applied. For example, it includes a dedicated system for generating a styling display image, or a system that is generally used (as an example, a general-purpose computer, a server computer, or a plurality of computers distributed on a network (that is, a so-called cloud computer), etc.). It is a concept that includes a system realized by installing a generation program on such a system to implement a function for generating a styling display image.

[0020] The "styling target object" is an object applied to the application target person, and includes, for example, clothing, shoes, bags, various accessories (necklaces, rings, etc.), and cosmetics.

[0021] The "application target person" is a target to which the styling target object is applied, and includes, for example, humans and animals (pets such as dogs and cats).

[0022] The "styling display image" is an image that displays the application target person in a state where the styling target object is applied.

[0023] And in the embodiments shown below, for example, when the "generation system" is applied to a system for realizing a so-called EC site (shopping site, etc.) that sells clothing and the like via the Internet, the case where the "styling target object" is clothing and the "application target person" is a human will be exemplified.

[0024] (Configuration) First, the configuration of the information system according to this embodiment will be described. FIG. 1 is a block diagram conceptually showing the information system according to this embodiment.

[0025] The information system 100 is a system including a generation system, for example, a system for realizing a so-called EC site for selling clothes or the like via the Internet. As an example, it includes a terminal device 1 and a server device 2.

[0026] (Configuration - Terminal Device) The terminal device 1 in FIG. 1 is a device used by a user (the target person), for example, a tablet terminal or a smartphone. As an example, it includes a communication unit 11, a touch pad 12, a display 13, a recording unit 14, and a control unit 15.

[0027] Note that as the terminal device 1, other terminal devices such as a personal computer can also be used. Also, the number of terminal devices 1 is arbitrary, but in this embodiment, the one shown in FIG. 1 will be cited for explanation.

[0028] (Configuration - Terminal Device - Communication Unit) The communication unit 11 is a communication means for communicating with external devices (for example, the server device 2, etc.). The specific type and configuration of this communication unit 11 are arbitrary, but for example, it can be configured using a known communication circuit or the like.

[0029] (Configuration - Terminal Device - Touch Pad) The touch pad 12 is an operation means for receiving various operation inputs from the user when pressed by the user's finger or the like. The specific configuration of this touch pad 12 is arbitrary, but for example, a known one equipped with operation position detection means such as a resistive film method or a capacitance method can be used.

[0030] (Configuration - Terminal Device - Display) The display 13 is a display means for displaying various images based on the control of the control unit 15. The specific configuration of this display 13 is arbitrary. For example, a flat panel display such as a known liquid crystal display or an organic EL display can be used. Note that the touch pad 12 and the display 13 may be integrally formed as a touch panel by overlapping each other.

[0031] (Configuration - Terminal Device - Recording Unit) The recording unit 14 is a recording means for recording programs and various data necessary for the operation of the terminal device 1, and can be configured using, for example, a flash memory or the like (the recording units of other devices are the same).

[0032] (Configuration - Terminal Device - Control Unit) The control unit 15 is a control means for controlling the terminal device 1. Specifically, it is a computer configured to include a CPU, various programs interpreted and executed on the CPU (including basic control programs such as an OS and application programs that are launched on the OS and realize specific functions), and an internal memory such as a RAM for storing programs and various data (the control units of other devices are the same). In particular, the program according to the embodiment is installed in the terminal device 1 via an arbitrary recording medium or network, thereby substantially constituting each part of the control unit 15 (the control units of other devices are the same).

[0033] (Configuration - Server Device) The server device 2 is a generation system, and includes, for example, a communication unit 21, a recording unit 22, and a control unit 23.

[0034] Note that all or part of the various functions for realizing the EC site may be implemented in this server device 2, or may be implemented in another server device that can communicate with the server device 2.

[0035] Also, as a configuration for realizing the various functions of the EC site, any configuration including known configurations can be applied, so the description of the configuration is basically omitted.

[0036] (Configuration - Server Device - Communication Unit) The communication unit 21 in FIG. 1 is a communication means for communicating with an external device (for example, the terminal device 1, etc.). The specific type and configuration of this communication unit 21 are arbitrary, but for example, it can be configured using a known communication circuit or the like.

[0037] (Configuration - Server Device - Recording Unit) The recording unit 22 in FIG. 1 is a recording means (storage means) for recording programs and various data necessary for the operation of the server device 2, and is configured using, for example, a hard disk or a flash memory (not shown) as an external recording device. However, instead of or together with the hard disk or the flash memory, any other recording medium can be used, including a magnetic recording medium such as a magnetic disk, or an optical recording medium such as a DVD or a Blu-ray disk.

[0038] Stored in the recording unit 22 are, for example, as shown in FIG. 1, a base model image, obesity degree specification reference information, a first learned model on the generation side, a second learned model on the generation side, and a learned model on the styling side.

[0039] (Configuration - Server Device - Recording Unit - Base Model Image) FIG. 2 is a diagram illustrating a base model image.

[0040] The "base model image" is a reference application target person display image that displays the appearance of the reference application target person, and is, for example, an image that displays the base model B1 as shown in FIG. 2.

[0041] The "reference application target person" is a concept indicating a virtual application target person that serves as a standard for processing, and indicates a virtual target person whose body size and body shape, etc. are predetermined, and in the present embodiment, is also referred to as the "base model". The details of the base model image in FIG. 2 will be described later.

[0042] (Configuration - Server Device - Recording Unit - Obesity Degree Specific Criterion Information) The "Obesity Degree Specific Criterion Information" in FIG. 1 is information indicating criteria for specifying the obesity degree of a user who is the application target.

[0043] The "obesity degree" is a concept indicating the physical condition of a user, and is, for example, a concept defined according to BMI (Body Mass Index), and as an example, from a thin person to a fat person (that is, from a person with a smaller BMI value to a person with a larger BMI value), seven stages of states of being too thin, thin, standard, overweight, obesity degree 1, obesity degree 2, and obesity degree 3 may be used.

[0044] Note that the definition of the obesity degree and the number of stages of the state here are examples, and other definitions and other numbers of stages may be applied as long as they indicate the physical condition of the user.

[0045] And as the obesity degree specific criterion information in FIG. 1, for example, table information associating the range of BMI with each physical state (for example, BMI < 16.0 is "too thin", 16.0 ≤ BMI < 18.0 is "thin", etc.), or various programs or arithmetic expressions that output the obesity degree when height and weight (or the BMI value itself) are input may be used.

[0046] (Configuration - Server Device - Recording Unit - First Learned Model on the Generation Side) FIG. 3 is an explanatory diagram of each learned model. In FIG. 3, the input information ("Input" column) and output information ("Output" column) of each learned model are illustrated.

[0047] The "First Learned Model on the Generation Side" in FIG. 1 is a model after learning generated by machine learning performed in advance, and as illustrated in association with "Number" = "1" in FIG. 3, it is a model configured to generate and output a first user display image when a base model image, skeletal point information, and obesity degree information are input.

[0048] Note that the "obesity level information" is information indicating the obesity level described above. Also, the skeletal point information and the first user display image will be described later.

[0049] The specific machine learning method of this generation-side first learned model is arbitrary. For example, so-called supervised learning may be performed in accordance with a predetermined intention (the same applies to the generation-side second learned model and the styling-side learned model described later).

[0050] (Configuration - Server Device - Recording Unit - Generation-Side Second Learned Model) The "generation-side second learned model" in FIG. 1 is a model after learning generated by machine learning performed in advance. As exemplified in association with "number" = "2" in FIG. 3, when the first user display image, the first region specification information, and the face information are input, it is configured to generate and output a second user display image.

[0051] Note that the first user display image, the first region specification information, the face information, and the second user display image will be described later.

[0052] (Configuration - Server Device - Recording Unit - Styling-Side Learned Model) The "styling-side learned model" in FIG. 1 is a model after learning generated by machine learning performed in advance. As exemplified in association with "number" = "3" in FIG. 3, when the second user display image, the second region specification information, the product-related information, and the body size information are input, it is configured to generate and output a styling display image.

[0053] Note that the "styling display image" is, as described above, an image that displays the person to whom the styling target is applied in the applied state. In the present embodiment, it is an image that displays the user wearing the clothes selected by the user. Also, the second user display image, the second region specification information, the product-related information, and the body size information will be described later.

[0054] (Configuration - Server Device - Control Unit) The control unit 23 in FIG. 1 is a control means for controlling the server device 2.

[0055] Conceptually in terms of functions, this control unit 23 functions as, for example, an acquisition means, a specification means, a first generation means, and a second generation means.

[0056] ===Acquisition means=== The "acquisition means" is a means for acquiring body-related information indicating the body of the applicable person. The "acquisition means" further acquires, for example, styling object-related information indicating a styling object selected by the user of the generation system.

[0057] The "body-related information" is information indicating the body of the applicable person, and is, for example, information indicating height, weight, arm length, shoulder width, etc.

[0058] The "styling object-related information" is information indicating a styling object selected by the user of the generation system, and includes, for example, at least a styling object display image for displaying the styling object and styling object size information indicating the size of the styling object.

[0059] ===Specification means=== The "specification means" is a means for specifying reference point information indicating the positions of a plurality of reference points that serve as a basis for shaping the appearance of the applicable person's body, based on the body-related information acquired by the acquisition means, with reference to a reference applicable person display image that displays the appearance of the reference applicable person. Note that "with reference to the reference applicable person display image" may be interpreted, for example, as a concept indicating "based on the reference applicable person display image".

[0060] The "specification means" performs, for example, a first specification process for specifying the size of a predetermined part of the applicable person's body based on the body-related information acquired by the acquisition means, and a second specification process for specifying the reference point information based on the body-related information acquired by the acquisition means and the size of the predetermined part specified in the first specification process.

[0061] The "specific means" specifies the obesity information of the person to whom the application is to be applied, for example, based on the body-related information acquired by the acquisition means.

[0062] ===First generation means === The "first generation means" is a means for generating a display image of the person to whom the application is to be applied, which displays the appearance of the person to whom the application is to be applied, based on the body-related information acquired by the acquisition means.

[0063] The "first generation means" generates a display image of the person to whom the application is to be applied, for example, based on the reference point information specified by the specific means and the reference display image of the person to whom the application is to be applied.

[0064] The "first generation means" performs, for example, a first process on the first generation means side that generates a first display image of the person to whom the application is to be applied, which displays the appearance of the person to whom the application is to be applied, generated based on the whole body of the reference person to whom the application is to be applied, based on the reference point information specified by the specific means, the obesity information specified by the specific means, and the reference display image of the person to whom the application is to be applied; and a second process on the first generation means side that generates a second display image of the person to whom the application is to be applied, which displays the appearance of the person to whom the application is to be applied including the face corresponding to the face information, by editing only the face of the person to whom the application is to be applied shown in the first display image of the person to whom the application is to be applied generated in the first process on the first generation means side, based on the face information showing the face part of the person to whom the application is to be applied and input to the generation system.

[0065] ===Second generation means === The "second generation means" is a means for generating a styling display image based on the display image of the person to whom the application is to be applied generated by the first generation means and the styling object-related information indicating the styling object.

[0066] The "second generation means" generates a styling display image, for example, based on the size of a predetermined part of the body of the person to whom the application is to be applied specified in the first specific process, the display image of the person to whom the application is to be applied generated by the first generation means, and the styling object-related information.

[0067] The "second generation means" generates a styling display image, for example, based on the second target person display image generated by the first generation means and the styling object related information.

[0068] The "second generation means" generates a styling display image, for example, based on the target person display image generated by the first generation means, the styling object display image acquired by the acquisition means, and the styling object size information acquired by the acquisition means.

[0069] Note that the processing performed by each part of such a control unit 23 will be described later.

[0070] (Processing) Next, as the processing performed by the information system 100 configured as described above, for example, the image display processing will be described.

[0071] (Processing - Image Display Processing) FIG. 4 is a flowchart of the image display processing (hereinafter, each step is referred to as "S"). The image display processing is a process performed by the server device 2 and is generally a process of generating and displaying a styling display image.

[0072] The timing for executing this image display processing is arbitrary. For example, when a user who is the target person (hereinafter, also simply referred to as "user") performs a predetermined operation via his / her terminal device 1, it is activated and will be described from the point where the image display processing is activated.

[0073] Here, for example, in the case where application software for shopping on an EC site is installed in the user's terminal device 1, the case where the user inputs various information via the terminal device 1 will be exemplified and described.

[0074] As a variation, it may be configured such that installation of the application software is not required, and for example, by using a web site browsing software such as a browser, access to the server device 2 etc. is possible and shopping on the EC site becomes possible.

[0075] ===SA1=== In SA1 of FIG. 4, the control unit 23 of the server device 2 acquires body-related information (information indicating height, weight, arm length, and shoulder width respectively), facial information, and product-related information.

[0076] ==Body-related information== Specific processing for the body-related information is arbitrary. For example, when a user inputs information indicating their height, weight, arm length, and shoulder width respectively via their own terminal device 1, the information is transmitted from the terminal device 1 to the server device 2, and the server device 2 receives and acquires the transmitted information.

[0077] Here, for example, for height and weight, it is configured to input numerical information, and for arm length, it is configured to select and input one from options regarding sizes such as "short", "standard", "long", etc., and for shoulder width, it is configured to select and input one from options regarding sizes such as "narrow", "standard", "wide", etc.

[0078] And when the user inputs, as information indicating their height, weight, arm length, and shoulder width, for example, "145 cm", "55 kg", "standard", "wide", the information is transmitted from the terminal device 1 to the server device 2, and the server device 2 receives and acquires the transmitted information.

[0079] ==Facial information== "Facial information" is information indicating the user's face (the part above the neck including the face and head), and is a concept including, for example, a user's face photo or information indicating each part of the user's face (also referred to as "part information").

[0080] Note that the content of the "part information" is arbitrary. For example, images generated by any image software such as CAD other than photos, various types of information, etc. may be used. As an example, information indicating parts related to hair (parts representing length, color, bang type, hair texture, etc.) or information indicating skin color may be used.

[0081] Although the specific processing for facial information is arbitrary, when the user performs a first operation (an operation for inputting a face photo) via their own terminal device 1 and then takes a photo of their own face using a camera (not shown) of the terminal device 1, the facial information indicating the captured face photo of the user is transmitted from the terminal device 1 to the server device 2. Also, when the user performs a second operation (an operation for inputting part information) via their own terminal device 1 and then performs an operation of selecting part information indicating each part corresponding to themselves, the facial information indicating the selected part information is transmitted from the terminal device 1 to the server device 2. On the other hand, the server device 2 receives this transmitted facial information and acquires the received facial information.

[0082] ==Product-related information== "Product-related information" is information related to the styling object. Specifically, it is information indicating the product (clothes in this embodiment) selected by the user. For example, it includes a product image (an image of clothes in this embodiment) for displaying the product and product size information (the actual size of the clothes in this embodiment, including the overall size and dimensions of each part (in the case of a skirt, the overall sizes such as S, M, L, etc. and dimensions such as waist, hip, skirt length, etc.)) indicating the size of the product. Note that it may be interpreted that the product image corresponds to the "styling object display image" and the product size information corresponds to the "styling object size information".

[0083] Although the specific processing of product-related information is arbitrary, for example, it is assumed that the product-related information (product images and product size information) of each product sold on the EC site is recorded in the recording unit 22 of the server device 2. And when the user selects a product sold on the EC site via the terminal device 1 to conduct a purchase consideration within the EC site, the information indicating the selected product is transmitted from the terminal device 1 to the server device 2. On the other hand, the server device 2 receives this transmitted information and, based on the received information, acquires the product-related information (product image and product size information) corresponding to the product selected by the user from among the product-related information of each product recorded in the recording unit 22.

[0084] Here, for example, when the user selects a specific skirt from among the products sold on the EC site, the information indicating the selected skirt is transmitted from the terminal device 1 to the server device 2. On the other hand, the server device 2 receives this transmitted information and, based on the received information, acquires the product-related information (product image and product size information) corresponding to the specific skirt selected by the user from among the product-related information of each product recorded in the recording unit 22.

[0085] ===SA2=== In SA2 of FIG. 4, the control unit 23 of the server device 2 specifies the skeletal point information corresponding to the user based on the body-related information acquired in SA1 and the base model image of the recording unit 22.

[0086] ==Base Model Image== FIGS. 5 to 6 are diagrams illustrating the base model image, and FIG. 7 is an explanatory diagram of the reference skeletal points and skeletal points. In FIG. 5, each reference skeletal point is illustrated in the base model image of FIG. 2, and in FIG. 6, in addition to the information in FIG. 5, the coordinate system set in the base model image is illustrated.

[0087] As described above, the "base model image" in FIG. 2 is an image of a person to whom the standard is applied and is an image displaying the base model B1. The "base model" B1 is, as described above, a person to whom the standard is applied, and represents a virtual user whose body size, body shape, etc. are predetermined. For example, it is assumed that the height is 165 cm, the arm length is 70 cm, the shoulder width is 40 cm, the face width is 14.5 cm, the waist width is 35 cm, and so on.

[0088] ==Base Model Image - Size== Also, the size of the base model image is assumed to be a predetermined size (vertical 〇〇 pixels × horizontal △△ pixels). The display of the base model B1 in the base model image is set to an appropriate size (for example, as shown in FIG. 2, a size that allows a predetermined size of margin above, below, left, and right of the base model B1, and a size that can ensure good visibility when displayed and output) with respect to the size of the entire base model image.

[0089] ==Base Model Image - Standard Skeleton Points== Also, in the base model image as shown in FIG. 5, for example, 18 standard skeleton points from the first standard skeleton point 901 to the 18th standard skeleton point 918 (refer to the "Standard Skeleton Points" column in FIG. 7) are defined.

[0090] In addition, the standard skeleton points are collectively referred to simply as "standard skeleton points". The standard skeleton points exemplified here are for convenience of explanation. For example, a large number of standard skeleton points may be defined for the hand part of the base model B1, or standard skeleton points may be defined for other parts, or some of the standard skeleton points may be omitted.

[0091] The "reference skeleton points" are a plurality of reference points that form the basis for the outer shape of the body of the base model B1. Specifically, it is a concept that includes positions corresponding to predetermined parts such as joints of the body. In this embodiment, as the reference skeleton points, the 1st reference skeleton point 901 to the 18th reference skeleton point 918 corresponding to the positions corresponding to the "parts" in FIG. 7 are defined. That is, for example, the 1st reference skeleton point 901 corresponding to the "nose" of the base model B1, the 2nd reference skeleton point 902 corresponding to the "right eye", etc. are defined.

[0092] ==Base Model Image - Coordinate System== Also, in the base model image, as shown in FIG. 6, for example, a coordinate system composed of the X-axis (horizontal axis) and the Y-axis (vertical axis) is defined. For each position in the base model image (that is, the positions of reference skeleton points, etc.), it can be specified by the coordinates based on this coordinate system.

[0093] Regarding the coordinate system in FIG. 6 in detail, for example, for the X-axis, coordinates with increasing numerical values from the left side to the right side are defined, and for the Y-axis, coordinates with increasing numerical values from the upper side to the lower side are defined. The values of each coordinate can be set based on, for example, the pixels of the base model image. Also, as a variation, other coordinate systems may be used.

[0094] ==Skeleton Point Information== "Skeleton point information" (not shown in FIGS. 5 and 6) is reference point information. Specifically, it is information indicating the positions of skeleton points (reference points).

[0095] The "skeleton points" are a plurality of reference points that form the basis for the outer appearance of the body of the person to whom the application is targeted. Specifically, they are a plurality of reference points that form the basis for the outer appearance of the user model indicating the user (specifically, the 1st user model U1 and the 2nd user model U2 described later). It is a concept that includes positions corresponding to predetermined parts such as joints of the body.

[0096] ==Processing== Although the processing of SA2 in FIG. 4 is optional specifically, for example, the following first processing (first specifying processing) and second processing (second specifying processing) are performed.

[0097] =First Processing= The first processing is, generally speaking, a process of specifying the size of a predetermined part of the user's body based on the body-related information acquired by SA1. Note that the predetermined part for which the size is specified is a part necessary for specifying the skeletal point information indicating the positions of the skeletal points corresponding to the parts of the reference skeletal points in FIG. 7 (18 parts in FIG. 7).

[0098] Here, for example, the case of specifying the sizes of the shoulder width and the waist width as the sizes of a predetermined part of the user's body will be specifically exemplified and described.

[0099] First, regarding the shoulder width, for example, in the body-related information acquired by SA1, as described above, it is information in three stages of "narrow", "standard", and "wide". However, based on the height and "narrow", "standard", and "wide", an arithmetic formula or the like for specifying the specific size (such as ○○ cm, etc.) of the shoulder width is predetermined, and based on this arithmetic formula, for example, 38 cm is specified as the specific size.

[0100] Also, regarding the waist width, for example, since the waist width is not included in the body-related information acquired by SA1, based on the height and shoulder width indicated by the body-related information, an arbitrary specifying method (for example, table information or an arithmetic formula showing the relationship between the height and shoulder width and the waist width is predetermined, and a method of specifying using this predetermined table information or arithmetic formula, etc.) is used to specify ○○ cm as the size of the waist width.

[0101] Also, in addition to the shoulder width and the waist width, the sizes of the parts necessary for specifying the skeletal point information indicating the positions of the skeletal points corresponding to the parts of the reference skeletal points in FIG. 7 (18 parts in FIG. 7) (for example, the length of the arm, the width of the face, and other parts, etc.) are specified by an arbitrary method (for example, a method of using a predetermined arithmetic formula for specifying the size of each part from the body-related information acquired by SA1, etc.).

[0102] =Second Process The second process is, generally speaking, a process of specifying skeletal point information based on the body-related information acquired by SA1 and the size of the predetermined part specified in the aforementioned first process.

[0103] In this second process, for example, in an image of the same size as the base model image, the position (coordinates) of each reference skeletal point information in FIGS. 6 and 7 is used to specify the skeletal point information so that a user model (specifically, the first user model U1 and the second user model U2 described later) showing the user in an appropriate size is displayed.

[0104] Specifically, the following first step to second step are executed.

[0105] <<<First Step>>> In the first step, the coordinates (that is, the positions) of the reference skeletal points defined in the base model image are specified. Here, for example, based on the coordinate system set in the base model image of FIG. 6, the coordinates of each of the first reference skeletal point 901 to the eighteenth reference skeletal point 918 are specified.

[0106] <<<Second Step>>> In the second step, based on the processing result in the first step, the coordinates (that is, the positions) of each skeletal point corresponding to each part of the reference skeletal point are specified as each skeletal point information. Note that the skeletal point corresponding to the part of the first reference skeletal point 901, which is the "nose", is referred to as the "first skeletal point", and similarly, each skeletal point corresponding to each part of the second reference skeletal point 902 to the eighteenth reference skeletal point 918 is referred to as the "second skeletal point" to the "eighteenth skeletal point" for explanation.

[0107] Generally speaking, based on the coordinates of each reference skeletal point, the coordinates of the skeletal point corresponding to the reference skeletal point are specified. That is, for example, based on the coordinates of the first reference skeletal point 901, the coordinates of the first skeletal point (the skeletal point corresponding to the "nose" of the user) corresponding to the first reference skeletal point 901 are specified.

[0108] <<Fixed>> Although it is optional in detail, for those described as "fixed" in the "X coordinate" and "Y coordinate" columns of FIG. 7, the same coordinates as those of the reference skeleton points are specified as the coordinates of the skeleton points.

[0109] Here, for example, for the first skeleton point where "part" in FIG. 7 = "nose", since both the X coordinate and the Y coordinate associated with the "first reference skeleton point 901" (in the "reference skeleton point" column of FIG. 7) corresponding to the first skeleton point are "fixed", the same XY coordinates as the XY coordinates of the first reference skeleton point 901 specified in the first step are specified as the coordinates of the first skeleton point. That is, the coordinates indicating the same position as the first reference skeleton point 901 in FIG. 6 and the like become the position of the first skeleton point corresponding to the "nose" of the user.

[0110] Also, for example, for the fifth skeleton point where "part" in FIG. 7 = "right shoulder", since only the Y coordinate is "fixed", the same Y coordinate as the Y coordinate of the fifth reference skeleton point 905 specified in the first step is specified as the Y coordinate of the fifth skeleton point. The method for specifying the X coordinate of the fifth skeleton point will be described later.

[0111] And for the coordinates of the skeleton points of other parts in FIG. 7 that are "fixed", they are specified in the same way.

[0112] <<Other than fixed>> Also, for those other than those described as "fixed" (blank or "corresponding to the right side") in the "X coordinate" and "Y coordinate" columns of FIG. 7, a predetermined calculation is performed, and the coordinates of the skeleton points are specified by reflecting the calculation result in the coordinates of the reference skeleton points.

[0113] The content of the predetermined calculation may be determined for each part. However, the calculation examples of the X coordinates of the fifth skeleton point and the thirteenth skeleton point where "part" in FIG. 7 = "right shoulder" and "left shoulder" will be described in detail, and only an overview will be described for other skeleton points.

[0114] <X coordinates of right shoulder and left shoulder - other than fixed> FIGS. 8 and 9 are explanatory diagrams of the calculation examples of the fifth skeleton point and the thirteenth skeleton point. FIG. 8 corresponds to a partially enlarged view of FIG. 6.

[0115] As described in the "<Premise>" column of FIG. 9, the number of pixels corresponding to the height of the base model B1 in the base model image of FIG. 6 (that is, the number of pixels in the Y-axis direction corresponding to the length L of FIG. 6, also referred to as the "base model height pixel number") is 890 pixels (890 pixels), and it is assumed that the body shape of the base model is determined as a height of 165 cm, an arm length of 70 cm, a shoulder width of 40 cm, a face width of 14.5 cm, a waist width of 35 cm, etc. as described above. Also, as the user's body shape, based on the body-related information acquired at SA1 and the processing result in the "<First Process>", it is assumed that a height of 145 cm, an arm length of 65 cm, a shoulder width of 38 cm, a face width of XX cm, a waist width of XX cm, etc. are specified.

[0116] Then, the following calculations are performed using these values.

[0117] Specifically, the first conversion ratio is calculated by performing the calculation shown in the arithmetic expression illustrated in the upper part of the "<Conversion Ratio>" in FIG. 9. The "first conversion ratio" is a concept indicating the number of pixels per 1 cm of the height of the base model B1. Here, for example, a calculation of 890 (pixels) ÷ 165 cm = 5.4 is performed.

[0118] Also, the second conversion ratio is calculated by performing the calculation shown in the arithmetic expression illustrated in the lower part of the "<Conversion Ratio>" in FIG. 9. The "second conversion ratio" is a concept indicating the number of pixels per 1 cm of the user's height. Here, for example, a calculation of 890 (pixels) ÷ 145 cm = 6.14 is performed.

[0119] Next, the X coordinate of the fifth skeleton point is calculated by performing the calculation shown in the arithmetic expression illustrated in the upper part of the "<Coordinate Calculation>" in FIG. 9. Note that "(shoulder width (base))" and "(shoulder width (user))" indicate the shoulder width of the base model B1 and the shoulder width of the user. For example, as shown in FIG. 8, when the coordinates of the fifth reference skeleton point 905 are (200, Ys5), by performing the calculation illustrated in FIG. 9, 191 is calculated and specified as the X coordinate of the fifth skeleton point.

[0120] In this case, as shown in FIG. 7, the Y coordinate of the fifth skeleton point is "fixed" and is the same as the Y coordinate of the fifth reference skeleton point. Therefore, as shown in FIG. 8, the coordinates of the fifth skeleton point are calculated and specified as (191, Ys5).

[0121] Also, the X coordinate of the 13th skeleton point is calculated by performing the calculation shown in the arithmetic expression illustrated in the lower part of "<Coordinate Calculation>" in FIG. 9.

[0122] Here, as described in the "X coordinate" column of the 13th reference skeleton point 913 in FIG. 7 as "corresponding to the right side", it is calculated in basically the same way as the coordinates of the fifth skeleton point which is the right shoulder corresponding to the left shoulder. However, from the perspective of reflecting the left-right inversion based on the vertical axis center line in the base model B1 of FIG. 6, as the operator (the "+" part in "Xs13 (X coordinate of the 13th reference skeleton point 913)+(shoulder width (base)···" in FIG. 9), instead of "-" (refer to the arithmetic expression regarding the fifth skeleton point in the upper part of "<Coordinate Calculation>" in FIG. 9), "+" is used.

[0123] Regarding the coordinates of the skeleton points of other parts in FIG. 7 that "correspond to the right side", they are specified in the same way.

[0124] For example, as shown in FIG. 8, when the coordinates of the 13th reference skeleton point 913 are (300, Ys13), by performing the calculation illustrated in FIG. 9, 309 is calculated and specified as the Y coordinate of the 13th skeleton point.

[0125] In this case, as shown in FIG. 7, the Y coordinate of the 13th skeleton point is "fixed" and is the same as the Y coordinate of the 13th reference skeleton point 913. Therefore, as shown in FIG. 8, the coordinates of the 13th skeleton point are calculated and specified as (309, Ys13).

[0126] <X coordinates of the right hip and left hip other than fixed> The X coordinates of the 8th and 16th skeleton points corresponding to the right hip and left hip are also specified in the same way as above. In this case, the hip width may be used for calculation and specification instead of the base model B1 and the user's shoulder width.

[0127] <X coordinates of the right and left eyes other than fixed points> For the X coordinates of the second and eleventh skeleton points corresponding to the right and left eyes, they are specified in the same manner as above. In this case, instead of the base model B1 and the user's shoulder width, the distance between the two eyes may be used for calculation and specification. Note that the distance between the two eyes may be specified by performing a predetermined calculation based on the face width (for example, setting it to half the length of the face width, etc.).

[0128] <X coordinates of the right and left ears other than fixed points> For the X coordinates of the third and twelfth skeleton points corresponding to the right and left ears, they are specified in the same manner as above. In this case, instead of the base model B1 and the user's shoulder width, the face width may be used for calculation and specification.

[0129] <X coordinates of the right and left elbows other than fixed points> For the X coordinates of the sixth and fourteenth skeleton points corresponding to the right and left elbows, they are specified in the same manner as above. In this case, instead of the base model B1 and the user's shoulder width, the distance between the two elbows may be used for calculation and specification. Note that the distance between the two elbows may be specified by performing a predetermined calculation based on the shoulder width (for example, setting it to 1.2 times the length of the shoulder width, etc.).

[0130] <X coordinates of the right and left wrists, right and left knees, and right and left ankles other than fixed points> For the X coordinates of the seventh and fifteenth skeleton points corresponding to the right and left wrists, the ninth and seventeenth skeleton points corresponding to the right and left knees, and the tenth and eighteenth skeleton points corresponding to the right and left ankles, they are specified in the same manner as the X coordinates of the sixth and fourteenth skeleton points corresponding to the right and left elbows.

[0131] <Y coordinates of the right and left elbows, right and left wrists other than fixed points> Regarding the Y coordinates of the 6th and 14th skeleton points corresponding to the right and left elbows, and the 7th and 15th skeleton points corresponding to the right and left wrists, they may be specified by performing an operation similar to the operation of "<Coordinate calculation>" in FIG. 9 with respect to the Y axis. In this case, instead of the shoulder width, the length of half or the entire length of the arm may be used for the calculation. Although not shown in FIG. 7, these Y coordinates may also be set as "fixed" in the same manner as others.

[0132] Note that the method for specifying each skeleton point shown here is an example, and the coordinates set as "fixed" in FIG. 7 are also examples. Other methods may be used for specification, or the reference points set as "fixed" may be arbitrarily changed.

[0133] ===SA3=== In SA3 of FIG. 4, the control unit 23 of the server device 2 generates a first user display image based on the information acquired in SA1 and the skeleton point information (i.e., the coordinates of each skeleton point) specified in SA2.

[0134] ==First User Display Image== FIG. 10 is a diagram illustrating a first user display image. The "first user display image" is a first applicable person display image (applicable person display image), and specifically, it is an image that displays the appearance of the first user model U1 indicating the user (i.e., the appearance of the user). The first user display image can also be interpreted as an image that displays the appearance of the user generated based on the entire body of the base model B1, for example.

[0135] Also, the size of the first user display image may be set arbitrarily, but in this embodiment, for example, it is the same as the size of the base model image in FIG. 2 (i.e., a predetermined size of vertical 〇〇 pixels × horizontal △△ pixels). Also, for example, the sizes of the second user display image and the styling display image described later are also the same as the size of the base model image.

[0136] ==Processing== Although the processing of SA3 in FIG. 4 is arbitrary specifically, for example, the following first processing and second processing are performed.

[0137] =First Processing= The first processing is, generally speaking, processing for specifying the obesity level of the user. Specifically, based on the body-related information acquired by SA1, the height and weight of the user are specified, the BMI is calculated based on the specified height and weight, and with reference to the obesity level specification reference information in the recording unit 22 in FIG. 1, the obesity level corresponding to the calculated BMI is specified (that is, obesity level information is specified).

[0138] Here, for example, when the height and weight of the user are specified as "145 cm" and "55 kg", after calculating the corresponding BMI, "being slightly overweight" is specified. Note that the numerical values and processing results here are for convenience of explanation (the same applies to other results and numerical values).

[0139] =Second Processing= The second processing is, generally speaking, processing for generating a first user display image using the processing result, etc. of the first processing.

[0140] Specifically, the control unit 23 acquires the base model image (FIG. 6, etc.) in the recording unit 22 in FIG. 1, the skeleton point information specified by SA2 (that is, the coordinates of each skeleton point including the coordinates of the fifth skeleton point "(191, Ys5)" and the coordinates of the thirteenth skeleton point "309, Ys13" in FIG. 8), and the obesity level information indicating "being slightly overweight" which is the obesity level specified in the first processing, and inputs the acquired information into the generation-side first learned model (number in FIG. 3 = "1"). In this case, for example, the first user display image in FIG. 10 is output from the generation-side first learned model, that is, the control unit 23 generates the first user display image.

[0141] Since the first user model U1 represents the user as described above, it can be said that the first user display image in FIG. 10 is an image that displays the appearance of the user.

[0142] Also, as described above, the generation-side first pre-trained model has been pre-trained in machine learning through so-called supervised learning in accordance with the intention. Further, since the base model image, skeletal point information, and obesity information are input, as shown in FIG. 10, a first user display image (an image in which each skeletal point is reflected and which shows a slightly overweight state) that displays a first user model U1 that appropriately reflects the user's body based on the entire body of the base model B1 in FIG. 6 is generated.

[0143] Also, thereby, the first user model U1 is set to an appropriate size (for example, as shown in FIG. 10, a size that allows a margin of a predetermined size to be formed above, below, to the left, and to the right of the first user model U1 and that can ensure good visibility when displayed and output) with respect to the size of the entire first user display image (the same applies to the second user display image and the styling display image described later).

[0144] Note that the processing of this SA3 may be interpreted as corresponding to the "first processing on the first generation means side".

[0145] ===SA4=== In SA4 of FIG. 4, the control unit 23 of the server device 2 generates a second user display image based on the information acquired in SA1 and the first user display image and the like generated in SA3.

[0146] ==Second User Display Image== FIG. 11 is a diagram illustrating a second user display image. The "second user display image" is a second applicable person display image (applicable person display image), and specifically, it is an image that displays the appearance of a second user model U2 showing the user (that is, the appearance of the user). For example, unlike the first user display image, it is an image in which the face corresponding to the user is reflected.

[0147] ==Processing== Although the processing of SA4 in FIG. 4 is arbitrary specifically, for example, the following first processing and second processing are performed.

[0148] =First Processing= FIG. 12 is a diagram illustrating a first user display image. The first process is generally a process of identifying a first region M1 (FIG. 12) in the first user display image.

[0149] The “first region” M1 is a region corresponding to the face portion in the first user model U1 in the first user display image. Specifically, as shown in FIG. 12, it is a region surrounding the upper part (the part including the face and head) from the neck in the first user model U1.

[0150] This first region M1 is a region surrounding the non-masked portion (the portion intended to be edited (i.e., changed) in the following second process) in the first user display image. That is, the portion other than the first region M1 in the first user display image (the hatched portion in FIG. 12, also referred to as the “masked portion”) is a portion not intended to be edited (i.e., changed) in the following second process (i.e., a portion intended to be excluded from the object of change).

[0151] Regarding the process in detail, in the first user display image generated by SA3, using any method (a method using a known image processing program for recognizing each part of a person, a method of specifying based on the characteristics of each part, etc.), the face portion of the first user model U1 is identified, and the region surrounding the face portion is identified as the first region M1.

[0152] Here, for example, in the first user display image of FIG. 10 generated by SA3, the first region M1 of FIG. 12 is identified.

[0153] =Second Process= The second process is generally a process of generating a second user display image using the processing result of the first process and the like.

[0154] Specifically, the control unit 23 acquires the first user display image generated by SA3, the first region identification information indicating the first region M1 identified by the first process (for example, a coordinate group indicating the first region M1, etc.), and the face information acquired by SA1 (that is, the face information input to the server device 2), and inputs these acquired information to the generation-side second learned model (number in FIG. 3 = "2"). In this case, a second user display image such as that in FIG. 11 is output from the generation-side second learned model, that is, the control unit 23 generates the second user display image.

[0155] Note that, as described above, since the second user model U2 (FIG. 11) represents a user, it can be said that the second user display image in FIG. 11 is an image that displays the appearance of the user.

[0156] Note that, as described above, the generation-side second learned model has been machine-learned by so-called supervised learning in accordance with the intention, and since the first user display image, the first region identification information, and the face information are input, as shown in FIG. 11, an intention regarding the first region M1 in FIG. 12 (that is, an intention to display a face corresponding to the input face information with only the portion surrounded by the first region M1 as the editing target) is reflected, and a second user display image is generated.

[0157] That is, in the first user display image generated by SA3, by editing only the face of the user (specifically, the first user model U1) shown in the first user display image, a second applicable person display image that displays the appearance of the user (specifically, the second user model U2) including the face corresponding to the face information is generated.

[0158] Note that the process of SA4 may be interpreted as corresponding to the "second process on the first generation means side".

[0159] ===SA5=== In SA5 of FIG. 4, the control unit 23 of the server device 2 generates a styling display image based on the information acquired by SA1 and the second user display image generated by SA4, etc.

[0160] Specifically, although it is optional, for example, the following first process and second process are performed

[0161] =First Process= FIG. 13 is a diagram illustrating a second user display image. The first process is generally a process of specifying a second region M2 (FIG. 13) in the second user display image.

[0162] The "second region" M2 is a region corresponding to the part (wearing part) where the clothes selected by the user (in the case of FIG. 13, the skirt worn on the lower body part of the clothes) is worn in the second user display image. Specifically, when the clothes is a skirt, as shown in FIG. 13, it is a relatively wide region that surrounds with a margin the part where any type of skirt including a mini skirt and a long skirt is worn (the same applies to other types of clothes such as tops, jackets, and pants according to each type of clothes).

[0163] This second region M2 is a region that surrounds the non-mask part (the part intended to be edited (i.e., changed) in the following second process) in the second user display image. That is, the part other than the second region M2 in the second user display image (the hatched part in FIG. 13, also referred to as the "mask part") is a part not intended to be edited (i.e., changed) in the following second process (i.e., a part intended to be excluded from the object of change).

[0164] Regarding the process in detail, depending on the type of product, the wearing part (for example, the lower body part in the case of a skirt, the upper body part in the case of a jacket, etc.) is assumed to be predetermined, and the wearing part corresponding to the product indicated by the product-related information obtained by SA1 (that is, the product selected by the user for consideration of purchase within the EC site) is identified in the second user display image generated by SA4 using any method (a method using a known image processing program for recognizing each part of a person, a method for specifying based on the characteristics of each part, etc.), and the area that surrounds the wearing part with a margin is specified as the second area M2.

[0165] Here, for example, when in SA1, product-related information corresponding to a specific skirt selected by the user is obtained, the second area M2 in FIG. 13 is specified in the second user display image of FIG. 11 generated by SA4.

[0166] =Second Process= FIG. 14 is a diagram illustrating a styling display image. The second process is generally a process of generating a styling display image using the processing result of the first process and the like.

[0167] Specifically, the control unit 23 acquires the second user display image generated by SA4, the second area specification information indicating the second area M2 specified in the first process (for example, a group of coordinates indicating the second area M2), the product-related information acquired by SA1 (specifically, a product image and product size information), and body size information.

[0168] "Body size information" is information indicating the size of the user's body. For example, it is information indicating the size of the body specified based on the body-related information acquired by SA1. As an example, it is a concept including height and the same information as the size of a predetermined part of the user's body specified in the first process of SA2.

[0169] In particular, by using, as the "body size information", the same information as that used when specifying the skeleton point information in SA2 (such as information indicating the size of a predetermined part of the user's body specified in the first process of SA2, information indicating the user's height, etc.), it becomes possible to improve the processing accuracy here.

[0170] Here, for example, as the "body size information", a case will be exemplified and described where information indicating the user's height (height specified from the body-related information acquired in SA1), the length of the arm (length of the arm specified in the first process of SA2), the shoulder width (shoulder width specified in the first process of SA2), and the waist width (waist width specified in the first process of SA2) is used.

[0171] Then, the acquired information described above (that is, the second user display image, the second region specification information, the product-related information, and the body size information) is input to the styling-side learned model (number in FIG. 3 = "3"). In this case, for example, a styling display image as shown in FIG. 14 (an image displaying the second user model U2 in a state where the user has selected and worn a skirt) is output from the styling-side learned model. That is, the control unit 23 generates the styling display image.

[0172] Note that since the second user model U2 represents the user as described above, it can be said that the styling display image in FIG. 14 is an image displaying the user in a state where the user has selected and worn a skirt.

[0173] Also, as described above, the styling-side learned model has been machine-learned by so-called supervised learning in accordance with the intention. Further, since the second user display image, the second region specification information, the product-related information (specifically, the product image and the product size information), and the body size information are input, as shown in FIG. 14, the intention regarding the second region M2 in FIG. 13 (that is, only the part surrounded by the second region M2 is the editing target, and the product corresponding to the input product-related information is worn on the body of the input body size information) is reflected, and a styling display image is generated.

[0174] In particular, since product size information and body size information of product-related information are input, machine learning of the styling-side learned model can be performed in consideration of these, so that the body size and the product size can be reflected in the styling display image.

[0175] That is, for example, although not shown in this embodiment, when a user with a relatively tall height wears a skirt with a relatively small comparison size, the wearing state (such as the length of the skirt assumed to be located around the ankle is located around the knee, or the skirt assumed to be worn loosely is worn exactly right, etc.) can be reflected, so that a wearing state closer to the actual situation can be displayed.

[0176] ===SA6=== In SA6 of FIG. 4, the control unit 23 of the server device 2 transmits the styling display image generated in SA5 to the terminal device 1 of the user, and displays it on the display 13 of the terminal device 1.

[0177] FIG. 15 is an example of a display of a styling display image. Here, for example, by transmitting the styling display image of FIG. 14 generated in SA5, as shown in FIG. 15, the styling display image is displayed on the display 13 of the terminal device 1.

[0178] Then, based on the styling display image of FIG. 15, the user can check the wearing state of the skirt selected on the EC site and consider whether to purchase the skirt. Thus, the description of the image display process ends.

[0179] (Effects of the Embodiment) As described above, according to this embodiment, by generating a styling display image, for example, it is possible to allow a user to check a state in which a styling object is applied.

[0180] Also, by using the skeletal point information indicating the positions of a plurality of skeletal points serving as criteria for shaping the appearance of the body of the user who is the target of application, for example, the first user display image and the second user display image, the target-of-application user display image can be appropriately generated, and it becomes possible to appropriately confirm the state in which the clothing that is the styling target is applied.

[0181] Also, by generating a styling display image based on the size of a predetermined part of the user's body, for example, considering the size of a predetermined part of the user's body, the target-of-application user display image can be appropriately generated, and it becomes possible to appropriately confirm the state in which the clothing that is the styling target is applied.

[0182] Also, by performing a process of generating a second user display image based on the first user display image, for example, the second target-of-application user display image can be appropriately generated, and it becomes possible to appropriately confirm the state in which the clothing that is the styling target is applied.

[0183] Also, by generating a styling display image based on the product image and the product size information, for example, it becomes possible to appropriately confirm the state in which the clothing that is the styling target is applied.

[0184] [Modifications to the Embodiment] As described above, the embodiments according to the present invention have been described. However, the specific configurations and means of the present invention can be arbitrarily modified and improved within the scope of the technical idea of the present invention described in the claims. Hereinafter, such modifications will be described.

[0185] [Regarding the problems to be solved and the effects of the invention] First, the problems to be solved by the invention and the effects of the invention are not limited to the above-described content, and may vary depending on the implementation environment and details of the configuration of the invention. There may be cases where only some of the above-described problems are solved or only some of the above-described effects are achieved. Also, the features of the present application may be interpreted as being capable of solving other problems in addition to the explicitly stated problems.

[0186] (Regarding dispersion and integration) Furthermore, each of the above-described electrical components is a functional concept and does not necessarily have to be physically configured as shown in the drawings. That is, the specific forms of dispersion and integration of each part are not limited to those shown in the drawings, and all or part of them can be functionally or physically dispersed or integrated in any unit according to various loads and usage situations, etc. Also, the "device" in this application is not limited to being constituted by a single device, but includes those constituted by a plurality of devices.

[0187] (Regarding numerical values and time series) Regarding the components exemplified in the embodiments and drawings, with respect to the numerical values or the mutual relationship of time series, within the scope of the technical idea of the present invention, they can be arbitrarily modified and improved.

[0188] (Regarding the selection of products) Also, in the above embodiment, the case where only a skirt is selected is exemplified. However, when a plurality of products such as a skirt and a top are selected, the state of wearing the selected products will be displayed as a styling display image.

[0189] (Regarding body-related information) Also, in the above embodiment, the case where information indicating height, weight, arm length, and shoulder width is used as body-related information has been described, but it is not limited thereto. For example, only one type, only two types, or only three types of this information may be configured to be used as body-related information. Or, it may be configured to use information of other types (for example, waist circumference dimension, inside leg length, etc.).

[0190] (Regarding product-related information) Also, in the above embodiment, product feature information (information indicating the features of a product, for example, information indicating features related to materials such as having luster, or features related to the attributes of the persons targeted by the product such as for women in their 20s, etc.) may be used as part of the information constituting the product-related information.

[0191] (Regarding the learned model) Also, each learned model in FIG. 3 of the above embodiment may be arbitrarily changed. For example, by combining the learned models numbered "1" and "2", a learned model is generated that generates and outputs a second user display image when inputting a base model image, skeleton point information, obesity degree information, and face information, and the model is used to perform processing corresponding to SA3 and SA4 in FIG. 4. It may be configured as such.

[0192] Also, in each learned model in FIG. 3 of the above embodiment, some of the input information may be omitted. For example, in the learned model numbered "1", either one or both of the obesity degree information or the base model image may be omitted. Also, in the learned model numbered "3", the body size information may be omitted.

[0193] Also, in each process, instead of the learned model, it may be configured to use a predetermined processing algorithm (a program that does not use a learned model).

[0194] (Regarding the recommendation function) Also, for each of a plurality of users who use the information system 100 in FIG. 1, the styling display images generated in the past may be recorded in the recording unit 22. In this case, the server device 2 may be configured to output and display recommendation information regarding the wearing of clothing as a product to a specific user based on the recorded styling display images.

[0195] For example, assume that information indicating the height and weight of each user, the skeletal type (type classified from skeletal point information), and the personal color (color preferred by each user) is recorded in the recording unit 22. Then, based on the information about each user recorded in the recording unit 22, the control unit 23 of the server device 2 identifies a user corresponding to a specific user (the user for whom recommended information is to be output) (hereinafter referred to as the "corresponding user"), selects a styling display image generated in the past for the identified corresponding user, and may be configured to display the selected styling display image on the terminal device 1 of the aforementioned specific user.

[0196] In this case, the method for identifying the corresponding user is arbitrary. For example, users with the same height and weight may be identified, users with the same skeletal point type may be identified, or users with the same personal color may be identified. Alternatively, it may be configured to identify users in whom two or more of these height and weight, skeletal point type, or personal color are the same.

[0197] When there are a plurality of styling display images generated in the past, it may be configured to display all of the plurality of images, or alternatively, it may be configured to select and display a part of the plurality of images.

[0198] (Regarding other features) Also, in the above embodiment, it may be configured to use other types of information. For example, it may be configured to have the user input their own age and identify the skeletal point information in consideration of the age.

[0199] Also, it may be configured to be able to regenerate the second user display image of SA4 in FIG. 4. For example, after the processing of SA4, when it is configured to display the second user display image on the terminal device 1 of the user, and the user performs an update operation such as pressing an "update" button via the terminal device 1, the processing of SA4 may be performed again to regenerate the second user display image.

[0200] (Regarding combinations) Furthermore, the features of the above embodiments and the features of the modification examples may be arbitrarily combined.

[0201] (Appended Note) The generation system of Appended Note 1 is a generation system that generates a styling display image for displaying an applicable person in a state where a styling object is applied, and includes an acquisition means for acquiring body-related information indicating the body of the applicable person, a first generation means for generating an applicable person display image for displaying the appearance of the applicable person based on the body-related information acquired by the acquisition means, and a second generation means for generating the styling display image based on the applicable person display image generated by the first generation means and styling object-related information indicating the styling object.

[0202] The generation system of Appended Note 2 is the generation system according to Appended Note 1, and further includes a specifying means for specifying reference point information indicating the positions of a plurality of reference points that serve as a reference for shaping the appearance of the body of the applicable person, with reference to a reference applicable person display image that displays the appearance of a reference applicable person based on the body-related information acquired by the acquisition means. The first generation means generates the applicable person display image based on the reference point information specified by the specifying means and the reference applicable person display image.

[0203] The generation system of Appended Note 3 is the generation system according to Appended Note 2, wherein the specifying means performs a first specifying process for specifying the size of a predetermined part of the body of the applicable person based on the body-related information acquired by the acquisition means, and a second specifying process for specifying the reference point information based on the body-related information acquired by the acquisition means and the size of the predetermined part specified in the first specifying process. The second generation means generates the styling display image based on the size of the predetermined part of the body of the applicable person specified in the first specifying process, the applicable person display image generated by the first generation means, and the styling object-related information.

[0204] The generation system of Supplementary Note 4 is the generation system described in Supplementary Note 2. In this system, the specific means further specifies the obesity degree information of the applicable person based on the body-related information acquired by the acquisition means. The first generation means performs the following two processes: First, based on the reference point information specified by the specific means, the obesity degree information specified by the specific means, and the reference applicable person display image, it generates a first applicable person display image that displays the appearance of the applicable person generated based on the whole body of the reference applicable person. Second, for the facial information indicating the face of the applicable person, based on the facial information input into the generation system, in the first applicable person display image generated by the first process on the first generation means side, it edits only the face of the applicable person shown in the first applicable person display image, thereby generating a second applicable person display image that displays the appearance of the applicable person including the face corresponding to the facial information. The second generation means generates the styling display image based on the second applicable person display image generated by the first generation means and the styling object-related information.

[0205] The generation system of Supplementary Note 5 is the generation system described in Supplementary Note 1. In this system, the acquisition means further acquires the styling object-related information indicating the styling object selected by the user of the generation system. The styling object-related information includes at least a styling object display image that displays the styling object and styling object size information that indicates the size of the styling object. The second generation means generates the styling display image based on the applicable person display image generated by the first generation means, the styling object display image acquired by the acquisition means, and the styling object size information acquired by the acquisition means.

[0206] The generation program according to Supplementary Note 6 is a generation program that generates a styling display image for displaying an applicable person in a state where a styling target object is applied. The computer is caused to function as an acquisition unit that acquires body-related information indicating the body of the applicable person, a first generation unit that generates an applicable person display image for displaying the appearance of the applicable person based on the body-related information acquired by the acquisition unit, and a second generation unit that generates the styling display image based on the applicable person display image generated by the first generation unit and styling target object-related information indicating the styling target object.

[0207] (Effect of Supplementary Note) According to the generation system described in Supplementary Note 1 and the generation program described in Supplementary Note 6, by generating a styling display image, it becomes possible to, for example, confirm a state in which a styling target object is applied.

[0208] According to the generation system described in Supplementary Note 2, by using reference point information indicating the positions of a plurality of reference points that serve as a reference for shaping the appearance of the body of the applicable person, it becomes possible to, for example, appropriately generate an applicable person display image and appropriately confirm a state in which a styling target object is applied.

[0209] According to the generation system described in Supplementary Note 3, by generating a styling display image based on the size of a predetermined part of the body of the applicable person, it becomes possible to, for example, appropriately generate an applicable person display image in consideration of the size of a predetermined part of the body of the applicable person and appropriately confirm a state in which a styling target object is applied.

[0210] According to the generation system described in Supplementary Note 4, by performing a second process on the first generation means side that generates a second applicable person display image based on the first applicable person display image generated by the first process on the first generation means side, it becomes possible to, for example, appropriately generate the second applicable person display image and appropriately confirm a state in which a styling target object is applied.

[0211] According to the generation system described in Supplementary Note 5, by generating a styling display image based on the styling target object display image and the styling target object size information, for example, it becomes possible to appropriately confirm the state in which the styling target object is applied.

Explanation of Signs

[0212] 1 Terminal device 2 Server device 11 Communication unit 12 Touch pad 13 Display 14 Recording unit 15 Control unit 21 Communication unit 22 Recording unit 23 Control unit 100 Information system 901 First reference skeleton point 902 Second reference skeleton point 903 Third reference skeleton point 904 Fourth reference skeleton point 905 Fifth reference skeleton point 906 Sixth reference skeleton point 907 Seventh reference skeleton point 908 Eighth reference skeleton point 909 Ninth reference skeleton point 910 Tenth reference skeleton point 911 Eleventh reference skeleton point 912 Twelfth reference skeleton point 913 Thirteenth reference skeleton point 914 Fourteenth reference skeleton point 915 Fifteenth reference skeleton point 916 Sixteenth reference skeleton point 917 Seventeenth reference skeleton point 918 Eighteenth reference skeleton point B1 Base model L Length M1 First area M2 Second area U1 First user model U2 Second user model

Claims

1. A generation system for generating a styling display image that displays an applicable person in a state where a styling object is applied, comprising: an acquisition means for acquiring body-related information indicating the body of the applicable person; a first generation means for generating an applicable person display image that displays the appearance of the applicable person based on the body-related information acquired by the acquisition means; a second generation means for generating the styling display image based on the applicable person display image generated by the first generation means and styling object-related information indicating the styling object; a specifying means for specifying reference point information indicating the positions of a plurality of reference points serving as a reference for shaping the appearance of the body of the applicable person, with reference to a reference applicable person display image which is an image displaying the appearance of a reference applicable person, based on the body-related information acquired by the acquisition means; the first generation means generates the applicable person display image based on the reference point information specified by the specifying means and the reference applicable person display image; A generation system.

2. The reference point information is coordinates in a predetermined coordinate system in which the direction of the height of the reference applicable person in the reference applicable person display image is taken as a first direction, and the direction orthogonal to the first direction is taken as a second direction. The specifying means based on a first ratio which is the ratio of the size of the height of the reference applicable person to the number of pixels in the first direction corresponding to the height of the reference applicable person in the reference applicable person display image (hereinafter referred to as the height pixel number), and a first calculation result based on the size in the second direction at a predetermined part of the reference applicable person, specifies the coordinate in the second direction in the reference point information based on a second ratio which is the ratio of the size of the height of the applicable person to the height pixel number, and a second calculation result based on the size in the second direction at the predetermined part of the applicable person; The generation system according to Claim 1.

3. The specifying means performs a first specifying process for specifying the size of a predetermined part of the body of the applicable person based on the body-related information acquired by the acquisition means; and a second specifying process for specifying the reference point information based on the body-related information acquired by the acquisition means and the size of the predetermined part specified in the first specifying process. The second generation means generates the styling display image based on the size of a predetermined part of the body of the application target person specified in the first specific process, the application target person display image generated by the first generation means, and the styling target object related information. The generation system according to claim 1.

4. A generation system that generates a styling display image that displays an application target person in a state where a styling target object is applied, An acquisition means for acquiring body related information indicating the body of the application target person; A first generation means for generating an application target person display image that displays the appearance of the application target person based on the body related information acquired by the acquisition means; A second generation means for generating the styling display image based on the application target person display image generated by the first generation means and the styling target object related information indicating the styling target object; Based on the body related information acquired by the acquisition means, a specific means for specifying reference point information indicating the positions of a plurality of reference points that serve as a reference for shaping the appearance of the body of the application target person with reference to a reference application target person display image that displays the appearance of the reference application target person; The first generation means generates the application target person display image based on the reference point information specified by the specific means and the reference application target person display image; The specific means further specifies the obesity degree information of the application target person based on the body related information acquired by the acquisition means; The first generation means A first process on the first generation means side for generating a first application target person display image that displays the appearance of the application target person generated with reference to the entire body of the reference application target person based on the reference point information specified by the specific means, the obesity degree information specified by the specific means, and the reference application target person display image; A second process on the first generation means side for generating a second application target person display image that displays the appearance of the application target person including the face corresponding to the face information by editing only the face of the application target person shown in the first application target person display image generated in the first process on the first generation means side based on the face information indicating the face of the application target person and input to the generation system; The second generation means generates the styling display image based on the second application target person display image generated by the first generation means and the styling target object related information. Generation system.

5. The acquisition means further acquires the styling target-related information indicating the styling target selected by the user of the generation system, The styling target-related information includes at least a styling target display image for displaying the styling target, and styling target size information indicating the size of the styling target, The second generation means generates the styling display image based on the application target person display image generated by the first generation means, the styling target display image acquired by the acquisition means, and the styling target size information acquired by the acquisition means. The generation system according to any one of claims 1 to 4.

6. A generation program for generating a styling display image that displays an application target person in a state where a styling target is applied, causing a computer to an acquisition means for acquiring body-related information indicating the body of the application target person, a first generation means for generating an application target person display image that displays the appearance of the application target person based on the body-related information acquired by the acquisition means, a second generation means for generating the styling display image based on the application target person display image generated by the first generation means and the styling target-related information indicating the styling target, a specifying means for specifying the positions of a plurality of reference points serving as a reference for shaping the appearance of the body of the application target person, based on the body-related information acquired by the acquisition means and with reference to a reference application target person display image that is an image for displaying the appearance of a reference application target person; and function as The first generation means generates the application target person display image based on the reference point information specified by the specifying means and the reference application target person display image. Generation program.

7. A generation program for generating a styling display image that displays an application target person in a state where a styling target is applied, causing a computer to an acquisition means for acquiring body-related information indicating the body of the application target person, a first generation means for generating an application target person display image that displays the appearance of the application target person based on the body-related information acquired by the acquisition means, a second generation means for generating the styling display image based on the application target person display image generated by the first generation means and the styling target-related information indicating the styling target, Based on the body-related information acquired by the acquisition means, as specific means for specifying reference point information indicating the positions of a plurality of reference points serving as a reference for shaping the appearance of the applicable person, with reference to a reference applicable person display image that displays the appearance of the reference applicable person, Based on the reference point information specified by the specifying means and the reference applicable person display image, the first generation means generates the applicable person display image. Based on the body-related information acquired by the acquisition means, the specifying means further specifies the obesity degree information of the applicable person. The first generation means Based on the reference point information specified by the specifying means, the obesity degree information specified by the specifying means, and the reference applicable person display image, the first processing on the first generation means side for generating a first applicable person display image that displays the appearance of the applicable person generated with reference to the entire body of the reference applicable person. Based on the facial information indicating the face of the applicable person, and by editing only the face of the applicable person shown in the first applicable person display image generated by the first processing on the first generation means side based on the facial information input to the computer, the second processing on the first generation means side for generating a second applicable person display image that displays the appearance of the applicable person including the face corresponding to the facial information. Based on the second applicable person display image generated by the first generation means and the styling object-related information, the second generation means generates the styling display image. Generation program.

Images