Virtual try-on device and virtual try-on program

Abstract

To provide a virtual try-on technology that enables accurate evaluation of clothing.SOLUTION: A virtual try-on device includes a user information acquisition unit 52, a clothing information acquisition unit 54, a prediction condition acquisition unit 56, a prediction unit 62, and an output unit 46. The user information acquisition unit 52 acquires body information of a user. The clothing information acquisition unit 54 acquires clothing information related to the characteristics of a predetermined article of clothing. The prediction condition acquisition unit 56 acquires information related to the situation when performing a predetermined movement as prediction conditions. The prediction unit 62 predicts the state of the user when the user wears the predetermined article of clothing and performing the predetermined movement on the basis of the body information, the clothing information, and the prediction conditions. The output unit 46 outputs information related to the predicted user situation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to a virtual try-on device and a virtual try-on program. [Background technology]

[0002] There is a known virtual fitting technology that allows a user to virtually try on an item of wear, such as clothing, by generating a synthetic image of the user wearing the item of wear, without actually wearing the item. For example, Patent Document 1 describes a technology that uses information on the movement of the person trying on the clothing to estimate the movement of the clothing when the person tries on the clothing, and synthesizes an image of the clothing with the estimated movement onto a moving image of the person trying on the clothing. In this way, the technology described in Patent Document 1 presents the person trying on the clothing with a moving image that is close to the state when the person tries on the actual clothing. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2006-249618 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not take into consideration the conditions under which a person will exercise when actually wearing the garment. Therefore, there is room for improvement in the evaluation of garments.

[0005] The present disclosure has been made in consideration of these circumstances, and has a purpose to provide a virtual try-on technology that enables accurate evaluation of wearable items. [Means for solving the problem]

[0006] A virtual try-on device of one aspect of the present disclosure comprises a user information acquisition unit that acquires physical information of a user, a wearable item information acquisition unit that acquires wearable item information related to the characteristics of specified wearable items, a prediction condition acquisition unit that acquires information regarding the situation when performing a specified exercise as a prediction condition, a prediction unit that predicts the user's state when the user wears the specified wearable items and performs the specified exercise based on the physical information, the wearable item information, and the prediction conditions, and an output unit that outputs information regarding the user's state.

[0007] Another aspect of the present disclosure is a virtual fitting program that causes a computer to realize a function of acquiring body information of a user, a function of acquiring information on the characteristics of a predetermined item of wear, a function of acquiring information on a situation when performing a predetermined exercise as a prediction condition, a function of predicting a user's condition when the user wears the predetermined item of wear and performs the predetermined exercise based on the body information, the information on the item of wear, and the prediction condition, and a function of outputting information on the user's condition.

[0008] In addition, any combination of the above components, or mutual substitution of the components or expressions of the present invention among methods, devices, programs, temporary or non-temporary storage media storing programs, systems, etc., are also valid aspects of the present invention. Effect of the Invention

[0009] According to an aspect of the present disclosure, a virtual try-on technology that allows for accurate evaluation of wearable items can be provided. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a schematic diagram showing an overview of a virtual try-on system. [Diagram 2] 1 is a functional block diagram showing a schematic configuration of a virtual try-on system. [Diagram 3] 3 is a functional block diagram showing a schematic configuration of a calculation unit of the virtual try-on server. FIG. [Figure 4]4 is a functional block diagram showing a schematic configuration of a user information acquisition unit. FIG. [Diagram 5] FIG. 2 is a functional block diagram showing a schematic configuration of a prediction unit. [Figure 6] 13 is an example of a menu screen of a virtual fitting mode. [Figure 7] 13 is an example of a user information input screen. [Figure 8] 13 is an example of a selection screen for a face image included in an avatar. [Figure 9] 13 is an example of a selection screen for selecting a target wearable item. [Figure 10] 1 is an example of a display screen displaying avatar images. [Figure 11] 1 is an example of a display screen displaying avatar images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, an embodiment will be described with reference to the drawings. In the embodiment and the modified examples, the same or equivalent components are given the same reference numerals, and duplicated descriptions will be omitted as appropriate.

[0012] 1 is a schematic diagram showing an overview of a virtual try-on system 100. The virtual try-on system 100 mainly comprises a user terminal 10 and a virtual try-on server 20. The virtual try-on system 100 may further comprise a wearing goods information server 2, or the functions of the wearing goods information server 2 may be implemented within the virtual try-on server 20. Similarly, the virtual try-on system 100 may further comprise a reference motion information server 4, or the functions of the reference motion information server 4 may be implemented within the virtual try-on server 20. The components of the virtual try-on system 100 are connected to each other via wireless communication 16 and a network 18.

[0013] The user terminal 10 and the virtual try-on server 20 may each be a mobile terminal or computer including a central processing unit (CPU), a graphics processing unit (GPU), a random access memory (RAM), a read only memory (ROM), an auxiliary storage device, a display device, a communication device, etc., and a program stored in the mobile terminal or computer. For example, the user terminal 10 may be configured to use the program executed by the virtual try-on server 20 via a network 18. The virtual try-on system may be realized by a single device having the functions of the user terminal 10 and the virtual try-on server 20, and the user may execute the virtual try-on program by directly operating the device. The single device may be a personal computer, a mobile terminal such as a smartphone, or an information terminal such as a tablet terminal, or may be realized in the form of a terminal operated by an assistant such as a sales clerk installed in a store selling wearable products, and a program stored in the terminal.

[0014] In this specification, the term "virtual try-on device" may refer to the entire virtual try-on system 100, or may refer to the virtual try-on server 20. In this embodiment, the virtual try-on server 20 is provided with many of the characteristic functions of the "virtual try-on device," so the virtual try-on server 20 essentially corresponds to the "virtual try-on device." However, the characteristic functions of the "virtual try-on device" may be distributed between the user terminal 10 and the virtual try-on server 20, or may be provided with many of the functions in the user terminal 10.

[0015] An example of the operation of the virtual try-on system 100 will be described with reference to Fig. 1. Using the user terminal 10, the user inputs the user's own physical information, designation information for the wearable item that the user wishes to virtually try on (hereinafter referred to as the "target wearable item"), and information regarding the situation when performing the exercise (hereinafter referred to as the "target exercise") that is expected to be performed while wearing the target wearable item. The designation information for the target wearable item is, for example, product number information for the target wearable item. The information regarding the situation when performing the target exercise is used as a prediction condition for predicting the user's state, which will be described later.

[0016] The user terminal 10 transmits the user's physical information, the designated information of the target wearable item, and the predicted conditions to the virtual try-on server 20 via wireless communication 16 and the network 18. The virtual try-on server 20 acquires information on the characteristics of the target wearable item (hereinafter referred to as "wearable item information") from the wearable item information server 2. The virtual try-on server 20 predicts the user's condition when the user wears the target wearable item and performs a target exercise based on the acquired physical information, wearable item information, and predicted conditions. For example, when the target exercise is running, the user's condition includes information such as the breathability of the target wearable item and the strain on the body.

[0017] The virtual try-on server 20 models an avatar based on the user's physical information. The virtual try-on server 20 generates an avatar image that reflects the predicted user state in the modeled avatar. The virtual try-on server 20 transmits the generated avatar image to the user terminal 10. The user terminal 10 displays the received avatar image. In this way, the virtual try-on system 100 can display the user state when the user wears a specified try-on item and performs a specified exercise, by reflecting the user state in the avatar modeled based on the user's physical information.

[0018] Fig. 2 is a functional block diagram showing the schematic configuration of the virtual try-on system 100. Each functional block shown in each diagram including Fig. 2 can be realized in hardware terms by elements and mechanical devices such as a computer processor and memory, and in software terms by a computer program, etc., but here, functional blocks realized by the cooperation of these are depicted. These functional blocks can be realized in various forms by hardware, software, or a combination of both.

[0019] The user terminal 10 includes an operation processing unit 30, a display unit 32, a communication unit 34, and a storage unit 36. The operation processing unit 30 accepts user operations. The operation processing unit 30 accepts selection or input of, for example, the user's body information, designated information of the target wearable item, prediction conditions, and the like. The display unit 32 displays information received from the virtual fitting server 20, such as avatar images, evaluation information, and recommendation information, which will be described later. The operation processing unit 30 accepts, for example, operation instructions for information displayed on the display unit 32. In terms of hardware, the operation processing unit 30 and the display unit 32 may be integrated into one unit, for example, a touch panel. The storage unit 36 ​​stores, for example, the user's body information, exercise data when the user actually performs a predetermined exercise including the target exercise, and information regarding the user's preferences. Details of these pieces of information will be described later.

[0020] The communication unit 34 transmits information selected or input by the operation processing unit 30 and information stored in the memory unit 36 ​​to the virtual try-on server 20 via the network 18. The communication unit 34 receives information from the virtual try-on server 20 and sends it to the display unit 32. In terms of hardware, the communication unit 34 may be configured by a wireless communication module for wireless LAN communication, mobile phone communication, or the like.

[0021] The virtual try-on server 20 comprises a communication unit 40, a calculation unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives information transmitted from the user terminal 10 and sends it to the calculation unit 42, and also transmits information such as the results of calculations by the calculation unit 42 to the user terminal 10. In terms of hardware, the communication unit 40 may be configured by a communication module such as a wired LAN.

[0022] The calculation unit 42 executes calculation processing based on the information received by the communication unit 40. The details of the calculation processing executed by the calculation unit 42 will be described later. The calculation unit 42 stores the calculation result in the memory unit 44. The calculation result stored in the memory unit 44 is output from the output unit 46 and transmitted to the user terminal 10 via the communication unit 40.

[0023] The wearing goods information server 2 stores wearing goods information for one or more types of wearing goods, including the target wearing goods, in association with identification information such as an ID. The calculation unit 42 of the virtual trying-on server 20 requests the wearing goods information server 2 to transmit the wearing goods information of the target wearing goods via the communication unit 40, based on the designation information of the target wearing goods received from the user terminal 10. In response to the request from the virtual trying-on server 20, the wearing goods information server 2 transmits the wearing goods information of the target wearing goods to the virtual trying-on server 20. The wearing goods information will be described in detail later.

[0024] The reference motion information server 4 stores information (hereinafter referred to as "reference motion information") that defines standard motions of a person performing one or more types of exercise including a target exercise. The calculation unit 42 of the virtual try-on server 20 requests the reference motion information server 4 to transmit the reference motion information of the target exercise via the communication unit 40 based on information regarding the situation when performing the target exercise received from the user terminal 10, i.e., the prediction conditions. In response to a request from the virtual try-on server 20, the reference motion information server 4 transmits the reference motion information of the target exercise to the virtual try-on server 20. Details of the reference motion information will be described later.

[0025] 3 is a functional block diagram showing a schematic configuration of the calculation unit 42 of the virtual try-on server 20. The calculation unit 42 includes an information acquisition unit 50, a prediction unit 62, an avatar generation unit 70, an evaluation unit 64, and a recommendation unit 66.

[0026] The information acquisition unit 50 includes a user information acquisition unit 52, a worn item information acquisition unit 54, a prediction condition acquisition unit 56, a period acquisition unit 58, and a reference action acquisition unit 60. The user information acquisition unit 52 acquires information about the user of the user terminal 10.

[0027] 4 is a functional block diagram showing a schematic configuration of the user information acquisition unit 52. The user information acquisition unit 52 of the present embodiment includes a physical information acquisition unit 82, an exercise data acquisition unit 84, a preference information acquisition unit 86, and a feedback acquisition unit 88.

[0028] The physical information acquisition unit 82 acquires the user's physical information. The user's physical information includes, for example, the user's height, weight, body fat percentage, fasting or postprandial blood glucose level, body temperature, and information on the size and shape of the body part on which the target wearable item is worn. If the target wearable item is, for example, shoes or socks, the user's physical information may include foot shape information such as foot length and foot circumference. If the target wearable item is, for example, clothing, the user's physical information may include body type information. The user's physical information may include injury history information on the body part on which the target wearable item is worn. The physical information acquisition unit 82 may read image information such as a photo of the user's entire body or a specific body part, and identify the user's physical information from the image information.

[0029] The exercise data acquisition unit 84 acquires exercise data. The exercise data acquired by the exercise data acquisition unit 84 may be data recorded when the user actually performs the target exercise, data estimated from data recorded when the user performs an exercise other than the target exercise, or data estimated from the user's physical information. The target exercise may be any sport. In this specification, sports include relatively high-stress sports such as running, ball games, swimming, cycling, skiing, snowboarding, and skateboarding, as well as relatively low-stress exercises such as walking and stretching.

[0030] The exercise data may include various measurement data acquired when the user actually performs the target exercise. The exercise data may include, for example, data related to the movement measured by a motion sensor such as a 9-axis sensor. The exercise data may include biological data measured by a biological sensor such as a heart rate sensor or a body temperature sensor. The exercise data may include data such as position coordinates and altitude measured by a position information sensor such as a positioning sensor or an altitude sensor. The exercise data may include data such as temperature, air pressure, humidity, and wind speed measured by an environmental sensor such as a temperature sensor or an air pressure sensor. The exercise data may include weather information acquired via the Internet, etc. The exercise data is not limited to data obtained by a single measurement, but may be data measured multiple times when the user performs one target exercise, or data measured once or more during each target exercise when the user performs the target exercise multiple times.

[0031] When the target exercise is running or walking, the exercise data may include, for example, stride length, pitch (the number of steps per unit time, also known as cadence), the intensity of the landing impact, running or walking distance, time required, heart rate, maximum oxygen intake, form analysis values, running or walking route, and elevation gained.

[0032] The exercise data acquisition unit 84 may acquire the exercise data from an existing application that records the exercise data or a server that manages an Internet service.

[0033] The preference information acquisition unit 86 acquires information about the user's preferences. The information about the user's preferences is information about the user's preferences regarding the target wearable item, and may include, for example, information about whether the user prefers a larger or smaller size, information about whether the user prefers a firm or soft feel when the target wearable item is a shoe, information about the preferred color, information about the preferred design, and the like. The information about the user's preferences may include wearing history information that associates information about the wearable item that the user has worn in the past with information about the user's impressions of wearing the wearable item. The user's impressions may include information about the fit, information about parts of the wearable item that are likely to wear out, and the like. The information about the user's preferences may include information about the direction of performance that the user expects from the target wearable item, for example, information about whether the user values ​​functionality in the target sport or fashionability. The information about the user's preferences may be information input by the user using the user terminal 10, or may be information generated based on the user's past purchase history or search history of the wearable item, or may be information generated through a sensibility survey, a questionnaire, or the like. Information about a user's preferences may be inferred based on information about actions such as posts and activities on social media, and information about the content of accounts followed.

[0034] The feedback acquisition unit 88 acquires feedback information input by the user after actually wearing another article of wear different from the target article of wear and performing the target exercise. As will be described in detail later, the feedback information is used by the prediction unit 62 to predict the user's condition and can be used to improve the prediction accuracy.

[0035] Returning to FIG. 3, the wearable item information acquisition unit 54 acquires wearable item information relating to the characteristics of the target wearable item. The target wearable item is, for example, an item used for sports. The target wearable item includes shoes, shoe insoles, socks, taping, pressure wear, clothing, braces, supports, etc. The wearable item information may include information such as the shape of the target wearable item, the material for each part, mass, pressure at the wearing part, color, size, etc., as well as other information that may affect the performance of the target exercise. The wearable item information acquisition unit 54 may acquire wearable item information corresponding to the designation information (e.g., product number) of the target wearable item received from the user terminal 10 from the wearable item information server 2.

[0036] The prediction condition acquisition unit 56 acquires information on the situation when the target exercise is performed as a prediction condition. The prediction condition is used as a condition when the prediction unit 62 predicts the user's state. The prediction condition may include information on the environment when the target exercise is performed, information on the content of the exercise, etc. For example, if the target exercise is running, the prediction condition may include information on the running path, such as whether the road is paved or unpaved, weather information, wind speed information, running distance information, running pace information, etc. For example, if the target exercise is soccer, the prediction condition may include information on the surface type of the soccer field, such as whether the surface is natural grass, artificial grass, or soil.

[0037] The period acquisition unit 58 acquires information about a period. Although details will be described later, the period information acquired by the period acquisition unit 58 is used by the prediction unit 62 to generate post-wearing information. In other words, the period information acquired by the period acquisition unit 58 is used as information about a predetermined period when the prediction unit 62 predicts the post-exercise user state after the user wears the target garment and continues the target exercise for a predetermined period.

[0038] The reference motion acquisition unit 60 acquires reference motion information. The reference motion information is information on a model of motion determined for each type of target exercise, and for example, if the target exercise is running, it may be information on a human body model running in a standard form along the time axis. As will be described in detail later, the reference motion information is used as a reference motion when the prediction unit 62 predicts a user motion. The reference motion acquisition unit 60 may acquire, from the reference motion information server 4, reference motion information corresponding to the designation information of the target exercise received from the user terminal 10.

[0039] The prediction unit 62 predicts the user's condition when the user wears the target wear item and performs the target exercise based on the physical information acquired by the physical information acquisition unit 82, the wear item information acquired by the wear item information acquisition unit 54, and the prediction conditions acquired by the prediction condition acquisition unit 56.

[0040] The user state predicted by the prediction unit 62 may include information about various effects that the user will obtain by wearing the target wearable item and performing the target exercise. The user state may include information about the physical properties of the target wearable item, such as the breathability of the target wearable item, the pressure on the wearing part, temperature distribution, the burden on the body, the stress on the joints, the degree of muscle fatigue, the amount of sweat, and other information about the user's body. The user state is information that can be used to evaluate the target wearable item.

[0041] 5 is a functional block diagram showing a schematic configuration of the prediction unit 62. The prediction unit 62 of this embodiment includes a user movement prediction unit 90, a posture change prediction unit 92, a movement change prediction unit 94, and a wearable item change prediction unit 96.

[0042] The user movement prediction unit 90 predicts user movement when the user wears a target wearable item and performs a target exercise, based on the wearable item information of the target wearable item acquired by the wearable item information acquisition unit 54, the exercise data acquired by the exercise data acquisition unit 84, and the reference movement information acquired by the reference movement acquisition unit 60. The user movement predicted by the user movement prediction unit 90 is an example of a user state predicted by the prediction unit 62.

[0043] Specifically, first, the user motion prediction unit 90 predicts the motion of the user when performing the target motion based on the difference between the reference motion information and the motion data. Here, the motion data can be used as a parameter for predicting the user's unique motion by reflecting the user's habits and characteristics in the standard motion based on the reference motion information. Next, the user motion prediction unit 90 adjusts the predicted user motion based on the wearable item information of the target wearable item so that the predicted user motion is a motion performed while wearing the target wearable item. For example, if the target wearable item is a shoe and the mass of the shoe is larger than that of a normal shoe, the pitch of the user motion may decrease or the form may change. The degree of change in the user motion due to the wearable item information of the target wearable item may vary depending on the motion data. For example, the weaker the user's muscle strength, the more susceptible the user motion is to the mass of the target wearable item.

[0044] The posture change prediction unit 92 predicts a change in the user's posture when the user wears the target wearable item and continues the target exercise for a predetermined period of time. The movement change prediction unit 94 predicts a change in the user's movement when the user wears the target wearable item and continues the target exercise for a predetermined period of time. The wearable item change prediction unit 96 predicts a change in the target wearable item when the user wears the target wearable item and continues the target exercise for a predetermined period of time.

[0045] In this specification, the predetermined period may be a period acquired by the period acquisition unit 58, or may be a period set in advance. The predetermined period may be information on a period previously associated with at least one of the target wearable item and the target exercise. The predetermined period may vary depending on the target exercise, and may include a short period such as one exercise (one match, one race, etc.) to a long period such as several years. The above "continuing the target exercise for a predetermined period" is not limited to continuing the target exercise without a break, but includes habitually performing the target exercise. In addition, the predetermined period is not limited to a fixed period such as one hour, and may be a period until a certain condition is met, such as the period until one match is completed or the period until one race is completed.

[0046] The posture change prediction unit 92 predicts a change in the user's posture based on the reference motion information acquired by the reference motion acquisition unit 60, the exercise data acquired by the exercise data acquisition unit 84, and the wearable item information of the target wearable item acquired by the wearable item information acquisition unit 54. For example, the posture change prediction unit 92 predicts, based on the reference motion information and the exercise data, to what extent the user's posture will deteriorate when the user wears the target wearable item for a predetermined period and continues the target exercise. For example, the posture change prediction unit 92 predicts, based on the reference motion information, the exercise data, and the wearable item information of the target wearable item, the effect that a change in the support function of the target wearable item will have on a change in the user's posture after the user wears the target wearable item and continues the target exercise for a predetermined period.

[0047] The multiple parameters related to the exercise data and the multiple parameters related to the wearable item information of the target wearable item may change individually or in interaction with each other depending on the duration of the target exercise. For example, when the target exercise is running or walking, the exercise data includes a stride parameter and a pitch parameter. The stride parameter and the pitch parameter tend to decrease as the duration of the target exercise becomes longer. The user's exercise data also includes information on individual differences in the degree of change over time of each parameter. That is, the degree of change over time of the stride parameter and the degree of change over time of the pitch parameter differ depending on the user. The wearable item information of the target wearable item includes a mass parameter. The mass parameter is constant regardless of the duration of the target exercise. However, the larger the mass parameter, the more likely the user is to become fatigued, so the stride parameter and the pitch parameter, which are parameters of the exercise data, may decrease. In this way, the posture change prediction unit 92 can predict the change in the user's posture using information on the change over time of each parameter. The same applies to the prediction of the change in the user's action by the movement change prediction unit 94 and the prediction of the change in the target wearable item by the wearable item change prediction unit 96.

[0048] For example, when the target exercise is running, the change in the user's posture is a change in the running form, and the posture change prediction unit 92 predicts, for example, how much the hips will sag and whether the upper body will lean forward at the end of a long-distance race such as a marathon. The posture change prediction unit 92 may predict, as the change in the user's posture, a change in the shape of the part of the body where the target wearable item is worn, for example, the degree of swelling of the feet if the target wearable item is shoes. The posture change prediction unit 92 may predict, based on the reference motion information, the exercise data, and the wearable item information of the target wearable item, a change in the user's facial expression, etc., caused by the user continuing the target exercise for a predetermined period while wearing the target wearable item.

[0049] The motion change prediction unit 94 predicts a change in the user's motion based on the reference motion information acquired by the reference motion acquisition unit 60, the exercise data acquired by the exercise data acquisition unit 84, and the wearable item information of the target wearable item acquired by the wearable item information acquisition unit 54. For example, the motion change prediction unit 94 predicts a change in the user's motion due to fatigue caused by the user continuing the target exercise while wearing the target wearable item for a relatively short period of time, based on the reference motion information and the exercise data. For example, the motion change prediction unit 94 predicts the influence on the user's motion of the user's habituation or familiarity with the target wearable item after the user continues the target exercise while wearing the target wearable item for a relatively long period of time, based on the reference motion information, the exercise data, and the wearable item information of the target wearable item.

[0050] The wearable item change prediction unit 96 predicts the above-mentioned change in the target wearable item based on the reference motion information acquired by the reference motion acquisition unit 60, the exercise data acquired by the exercise data acquisition unit 84, and the wearable item information of the target wearable item acquired by the wearable item information acquisition unit 54. For example, the wearable item change prediction unit 96 predicts a change in the elasticity of the target wearable item and a change in the properties such as the support force at the contact part with the user's body caused by the user wearing the target wearable item for a predetermined period and continuing the target exercise based on the reference motion information, the exercise data, and the wearable item information of the target wearable item. If it is considered that the target wearable item will not change significantly in a short period of time, the wearable item change prediction unit 96 may predict that the target wearable item will not change if the predetermined period is equal to or less than a certain threshold value.

[0051] As described above, the posture change prediction unit 92, the movement change prediction unit 94, and the wearable item change prediction unit 96 respectively predict the change in the user's posture, the change in the user's movement, and the change in the wearable item caused by the user wearing the wearable item and continuing the wearable item for a predetermined period of time. The prediction unit 62 predicts the post-exercise user state after the user wears the wearable item and continues the wearable item for a predetermined period of time based on at least one of the prediction results of the change in the user's posture, the change in the user's movement, and the change in the wearable item caused by the user wearing the wearable item and continuing the wearable item for a predetermined period of time. The post-exercise user state is an example of the user state. The post-exercise user state may be the same as the information of the prediction result described above. The prediction unit 62 may predict the gradual change in the user state when the user continues the wearable item for a predetermined period of time as the post-exercise user state. For example, if the wearable item is running and the predetermined period is one marathon, the prediction unit 62 may predict the change in the user state every time the user runs 5 km as the post-exercise user state.

[0052] The post-exercise user state may include information on performance indices for the target exercise. For example, if the target exercise is running, the performance indices may be analysis values ​​related to running form, predicted values ​​for race time, etc. The performance indices are not limited to absolute values, but may be relative values ​​that indicate how much the performance indices change in comparison with other wearable items.

[0053] The post-exercise user condition may include a fit indicator that indicates how well the subject article fits the user.

[0054] The prediction unit 62 may predict the user's condition further based on the feedback information acquired by the feedback acquisition unit 88. As described above, the feedback information is information related to feedback input by the user after actually wearing an article of wear other than the target article of wear and performing the target exercise. In this way, by the prediction unit 62 predicting the user's condition further based on the feedback information, the feedback information supplements the user's physical information, thereby improving the accuracy of the prediction.

[0055] The user movement prediction unit 90 may correct the predicted user movement based on at least one of the predicted results of the change in the user's posture predicted by the posture change prediction unit 92, the change in the user's movement predicted by the movement change prediction unit 94, and the change in the target worn item predicted by the worn item change prediction unit 96. In this way, the user movement prediction unit 90 can predict the user movement after the user wears the target worn item and continues the target exercise for a predetermined period of time.

[0056] The prediction by the prediction unit 62 may use a prediction model that has been trained by machine learning. In this case, when the prediction model receives the body information, the wearable item information, the prediction condition, the exercise data, the reference motion information, the period information, and the feedback information, it outputs the user state including the user motion and the user state after exercise. Furthermore, the prediction by the prediction unit 62 may use an analysis method such as regression analysis or multivariate analysis. In this case, the body information, the wearable item information, the prediction condition, the exercise data, the reference motion information, the period information, and the feedback information may be explanatory variables, and the user state including the user motion and the user state after exercise may be objective variables.

[0057] Returning to FIG. 3, the avatar generation unit 70 includes a modeling unit 72 and an image generation unit 74. The modeling unit 72 models an avatar based on the user's physical information. The avatar is a character that is expressed in a virtual space as an alter ego of the user, and can be modeled using known technology. The avatar may represent the user's entire body, a specific body part of the user, for example, a body part that wears the target wearable item, or both. The avatar may represent the user realistically, may represent the user with the user's features emphasized, may be simplified using stick pictures, or may be represented as a character such as an animal. The modeling unit 72 may model a part of the avatar, for example, a face type, based on information selected by the user.

[0058] The modeling unit 72 reflects the user's physical information in the avatar. For example, if the avatar represents the user's entire body, the modeling unit 72 models the avatar with a body type according to the user's height, weight, body fat percentage, etc. For example, if the avatar represents a specific body part of the user, the modeling unit 72 models the avatar according to the size of that body part, etc.

[0059] The modeling unit 72 may model the avatar based on the wearable item information of the target wearable item in addition to the user's body information. In this case, the modeling unit 72 models the avatar in a state where the target wearable item is worn. The avatar may change depending on the type, size, shape, color, etc. of the target wearable item.

[0060] The image generation unit 74 generates an avatar image that reflects the user state predicted by the prediction unit 62 in the avatar modeled by the modeling unit 72. For example, when the user state is information about the physical properties of the target wearable item, such as breathability, pressure, temperature distribution, etc., the image generation unit 74 generates an avatar image in which the display mode, such as the color distribution of the body parts wearing the target wearable item, is changed. For example, when the user state is information about the user's body, such as muscle fatigue level, amount of heat generation, etc., the image generation unit 74 generates an avatar image in which the display mode, such as the color distribution of the whole body and body parts, and the enlarged display of the target area, is changed.

[0061] When the user state predicted by the prediction unit 62 is a post-exercise user state, the image generation unit 74 generates an avatar image reflecting the post-exercise user state. The avatar image reflecting the post-exercise user state may have the same display form as the avatar image reflecting the above-mentioned user state. When the post-exercise user state predicted by the prediction unit 62 includes multiple gradual changes in the user state, the image generation unit 74 may generate avatar images corresponding to each change in the user state.

[0062] When the user action prediction unit 90 predicts the user action, the image generation unit 74 generates an avatar image so as to represent the situation when the avatar performs the target exercise with the user action. This allows the image generation unit 74 to reflect information on how the user moves when performing the target exercise while wearing the target wearable item in the avatar image.

[0063] The avatar image may be still image data, moving image data, or 3D modeled data. If the avatar image is 3D modeled data, the avatar image may be converted into a data format that allows the avatar image to be displayed when the avatar image is output by the output unit 46 and then displayed on the user terminal 10 or the like. The image generating unit 74 may take an image of the avatar seen from a third person's perspective as the avatar image, or may take an image of the perspective from the avatar as the avatar image. If the avatar image is an image of the avatar's perspective, for example, an image of a body part wearing the target wearable item may be taken, which makes it easier for the user to imagine the user's state when actually wearing the wearable item.

[0064] The evaluation unit 64 generates evaluation information of the target worn item based on the user state predicted by the prediction unit 62. The evaluation unit 64 may evaluate each of the information included in the user state information, such as information on the physical properties of the target worn item and information on the user's body, and generate evaluation information including an evaluation for each item and an overall evaluation. The evaluation information may include, for example, information on a score or rank according to the overall evaluation, and information on a score distribution according to the evaluation for each item. The evaluation information may be displayed as a graph or numerical values, or may be reflected in an avatar image by the above-mentioned image generation unit 74, and may change the facial expression of the avatar according to the overall evaluation, for example.

[0065] The recommendation unit 66 generates recommendation information indicating the wearable item recommended to the user based on the wearable item information for each of the multiple wearable items. In this case, the virtual fitting server 20 acquires wearable item information for each of the multiple wearable items, for example, by the following process. The wearable item information acquisition unit 54 acquires wearable item information for each of the multiple wearable items from the wearable item information server 2. Here, the wearable item information acquired by the wearable item information acquisition unit 54 may be wearable item information designated as a target wearable item, or wearable item information that satisfies a predetermined condition when not designated as a target wearable item. The prediction unit 62 predicts the user state for each of the multiple wearable items. The evaluation unit 64 generates evaluation information for each of the multiple wearable items based on the user state for each of the multiple wearable items. Based on the evaluation information for each of the multiple wearable items thus obtained, the recommendation unit 66 may generate information on the wearable item with the highest overall evaluation of the evaluation information as recommendation information that recommends the user to wear. The image generation unit 74 may generate avatar images that reflect the user state for each of the multiple wearable items predicted by the prediction unit 62. This allows avatar images for multiple items of wear to be displayed simultaneously, for example, making it easier to compare them.

[0066] 2, storage unit 44 stores the results of calculations, such as avatar images generated by calculation unit 42. Storage unit 44 may previously store information used by calculation unit 42 for calculations.

[0067] The output unit 46 outputs the avatar image generated by the avatar generation unit 70. The output unit 46 may output the information on the user state predicted by the prediction unit 62 in a form other than the avatar image, such as text, numerical value, or graph information. However, outputting the information in the form of an avatar image is preferable in that the user can intuitively grasp the user state. The output unit 46 may output the evaluation information generated by the evaluation unit 64. The output unit 46 may output the recommendation information generated by the recommendation unit 66. The information output by the output unit 46 is transmitted to the user terminal 10 via the communication unit 40.

[0068] 6 to 11 are examples of display screens displayed by the display unit 32 of the user terminal 10. Fig. 6 is an example of a menu screen in the virtual fitting mode. When the "Create Avatar" image 112 in Fig. 6 is selected, the display unit 32 transitions to the display screen in Fig. 7.

[0069] FIG. 7 is an example of a user information input screen. When the "Physical Information Input" image 124 in FIG. 7 is selected, the user terminal 10 accepts input of physical information such as body type and height. When the "Measurement Information" image 122 in FIG. 7 is selected, the virtual try-on server 20 obtains the user's athletic data, for example, from a server that manages a third-party service. When the "Generate" image 126 in FIG. 7 is selected, the user terminal 10 transmits the input physical information to the virtual try-on server 20, and the virtual try-on server 20 generates an avatar based on the physical information and athletic data.

[0070] Fig. 8 is an example of a selection screen for a facial image included in an avatar. As shown in Fig. 8, the user may be able to change the facial image included in the avatar.

[0071] When the "select shoes" image 114 in Fig. 6 is selected, the display unit 32 transitions to the display screen in Fig. 9. Fig. 9 is an example of a selection screen for selecting a target wearing item. In Fig. 9, shoes are displayed as target wearing items so that one or more shoes can be selected. The user terminal 10 transmits information designating the selected shoes as the target wearing items to the virtual fitting server 20.

[0072] Images corresponding to a plurality of types of exercise are displayed below the "Virtual Try-On" image 116 in Fig. 6. For example, when the running image 118 is selected, the user terminal 10 transmits information designating the target exercise as running to the virtual try-on server 20. The user terminal 10 further receives input of information relating to the situation when running as the target exercise is performed, and transmits this to the virtual try-on server 20 as a prediction condition.

[0073] The virtual fitting server 20, in the prediction unit 62, predicts the user's state when the user wears shoes designated as the target wearable items and runs designated as the target exercise, based on the input body information, wearable item information, and prediction conditions. The virtual fitting server 20 also generates an avatar image in the avatar generation unit 70 by reflecting the predicted user state in an avatar modeled based on the body information.

[0074] 10 and 11 are examples of display screens displaying the generated avatar images. The target wearable item for the avatar image in Fig. 10 is less tiring for the user, easier to stabilize the form, and more comfortable to wear than the target wearable item for the avatar image in Fig. 11, when the user performs the target exercise under the predicted conditions.

[0075] Here, a specific example using the virtual try-on system 100 of this embodiment will be described. First, a case where the target sport is soccer will be described. In this case, the user's body information includes height, weight, and foot shape information. The target wearable goods are shoes, and the wearable goods information includes the material and structure of the upper and outsole. The prediction conditions include information on the surface type of the soccer field, for example, whether the surface is natural grass, artificial grass, or soil. Note that in this example, the exercise data does not need to be used.

[0076] The prediction unit 62 in this example predicts information on the pressure applied to the feet and shoes during kicking and stepping movements as the user state. The avatar generation unit 70 in this example models at least one of the whole body and the feet as an avatar, and generates an avatar image showing the pressure information predicted by the prediction unit 62 by changing colors or the like. This makes it possible to predict the magnitude of the burden on the feet caused by the target wearable item.

[0077] Next, a case where the target exercise is running will be described. In this case, the user's physical information includes information on height, weight, and foot shape. The exercise data includes information on the environment in which the exercise was performed and the movements. The target worn items are shoes or running wear, and the worn item information includes information on size, material, and structure in the case of shoes. The material includes information on the upper material and sole material. The structure includes information on the last, midsole, shoelace, and upper structure. The worn item information includes information on size and fabric cutting pattern material in the case of sportswear. The prediction conditions include information on the type of running course, temperature, humidity, season, wind speed, running distance, running speed, type of running style, hunger level, and the like.

[0078] The prediction unit 62 in this example predicts information on muscle fatigue, strain on the body, and breathability of the running wear as the user's condition. In addition, when, for example, both shoes and running wear are selected as the target wearable items, the prediction unit 62 also includes the effect of their interaction in the user's condition. The avatar generation unit 70 in this example models at least one of the whole body and the feet as an avatar, and generates an avatar image showing the information predicted by the prediction unit 62 in the form of color changes, changes in the size of body parts, etc. This makes it possible to predict the wearing comfort of the target wearable item when used over a medium to long period of time.

[0079] As described above, according to this embodiment, the prediction unit 62 predicts the user's state when the user wears the target wearable item and performs the target exercise based on the body information, the wearable item information, and the prediction conditions. The avatar generation unit 70 generates an avatar image in which the user's state is reflected in an avatar modeled based on the body information. This makes it possible to generate an avatar image in which the user's state predicted according to the situation when performing a predetermined exercise is reflected. Therefore, the user can intuitively and accurately evaluate the wearable item by looking at the avatar image.

[0080] According to this embodiment, the prediction unit 62 predicts the user's motion when the user wears the target item and performs the target exercise based on the item information, the exercise data, and the reference motion information. The avatar generation unit 70 generates an avatar image so as to represent the situation when the avatar performs the target exercise with the user's motion. This generates an avatar image representing the user's motion that reflects the characteristics of the item, so the user can understand how the item affects the motion.

[0081] According to this embodiment, the prediction unit 62 predicts the user's post-exercise state after the user wears the target wearable item and continues the target exercise for a predetermined period as the user's state. In detail, the prediction unit 62 predicts at least one of a change in the user's posture, a change in the user's movement, and a change in the target wearable item caused by the user wearing the target wearable item and continuing the target exercise for a predetermined period, and predicts the user's post-exercise state based on the prediction result. This makes it possible to predict the state after the wearable item has been used for a certain period of time, and therefore to more accurately evaluate the wearable item.

[0082] The above-described embodiment may be a program for causing a computer to realize the functions for implementing the above-described method, or a recording medium for storing the program. Such a recording medium for storing the program may be a non-transitory and tangible computer-readable storage medium, and may be a non-volatile memory, a magnetic recording medium such as a magnetic tape or a magnetic disk, or an optical recording medium such as an optical disk.

[0083] The embodiment has been described above. This embodiment is merely an example, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each processing step, and that such modifications are also within the scope of the present invention. Furthermore, the above-described embodiment can be generalized to obtain the following aspects.

[0084] [Aspect 1] A user information acquisition unit that acquires physical information of a user; A wearable item information acquisition unit that acquires wearable item information related to characteristics of a predetermined wearable item; a prediction condition acquisition unit that acquires information regarding a situation when performing a predetermined exercise as a prediction condition; a prediction unit that predicts a user's state when the user wears the predetermined item of wear and performs the predetermined exercise based on the body information, the item of wear information, and the prediction conditions; an output unit that outputs information regarding the user state; A virtual try-on device comprising:

[0085] According to the present embodiment, the user's condition when wearing a specified item of wear and performing a specified exercise is predicted and output based on the user's physical information, information regarding the characteristics of the specified item of wear, and information regarding the circumstances when performing the specified exercise, so that the item of wear can be accurately evaluated from the outputted user condition.

[0086] [Aspect 2] an avatar generation unit that generates an avatar image in which the avatar modeled based on the physical information reflects the user's state; The output unit outputs the avatar image. 2. The virtual try-on device of embodiment 1.

[0087] According to this aspect, an avatar image is generated in which the user's state is reflected in an avatar based on the user's physical information and output, so that the user's state can be easily grasped intuitively from the output avatar image.

[0088] [Aspect 3] The user information acquisition unit further acquires exercise data when the user actually performs the predetermined exercise, A reference motion acquisition unit that acquires reference motion information that defines standard motions of a person performing the predetermined exercise, The prediction unit predicts a user's motion when the user wears the predetermined item of wear and performs the predetermined exercise based on the item of wear information, the exercise data, and the reference motion information; The avatar generation unit generates the avatar image so as to represent a situation in which the avatar performs the predetermined exercise through the user's action. 3. The virtual try-on device according to embodiment 2.

[0089] According to this aspect, the situation in which the user is wearing a predetermined item of clothing and performing a predetermined exercise can be reflected in the avatar image.

[0090] Aspect 4 The prediction unit predicts, as the user state, a post-exercise user state after the user wears the predetermined wearable item and continues the predetermined exercise for a predetermined period of time. 4. The virtual try-on device of embodiment 3.

[0091] According to this embodiment, the condition of the worn item after it has been used for a certain period of time can be predicted, so that the worn item can be evaluated with greater accuracy.

[0092] Aspect 5 the prediction unit predicts at least one of a change in the user's posture, a change in the user's movement, and a change in the specified item of wear caused by the user continuing the specified exercise for the specified period while wearing the specified item of wear, and predicts the user's state after exercise based on a prediction result; 5. A virtual try-on device according to embodiment 4.

[0093] According to this aspect, the condition of a worn item after it has been used for a certain period of time can be predicted based on predicted results of at least one of changes in the user's posture, changes in the user's movements, and changes in the worn item, thereby enabling the worn item to be evaluated with greater accuracy.

[0094] Aspect 6 The virtual try-on device according to aspect 4 or 5, wherein the post-exercise user state includes information on performance indicators of the specified exercise.

[0095] According to this embodiment, it is possible to predict information on athletic performance indicators after using the wearable item for a certain period of time, thereby enabling the wearable item to be evaluated with greater accuracy.

[0096] Aspect 7 A virtual try-on device according to any one of aspects 4 to 6, wherein the post-exercise user state includes information on the fit of the specified worn item.

[0097] According to this aspect, it is possible to predict the fit of the wearable item after the wearable item has been used for a certain period of time, thereby enabling the wearable item to be evaluated with greater accuracy.

[0098] Aspect 8 A period acquisition unit for acquiring period information is further provided, A virtual try-on device according to any one of aspects 4 to 7, wherein the prediction unit predicts the post-exercise state of the user using the period acquired by the period acquisition unit as the predetermined period.

[0099] According to this aspect, the condition of the worn item after it has been used for a specified period of time can be predicted, so that the worn item can be evaluated with greater accuracy.

[0100] Aspect 9 An evaluation unit that generates evaluation information of the predetermined item of wear based on the user state, 9. The virtual try-on device according to any one of aspects 1 to 8, wherein the output unit further outputs the evaluation information.

[0101] According to this aspect, evaluation information on the worn item is generated and output, so that the worn item can be evaluated more objectively.

[0102] Aspect 10 The wearing item information acquisition unit acquires the wearing item information of each of a plurality of wearing items, The prediction unit predicts the user state for each of the plurality of wearing items, The evaluation unit generates the evaluation information for each of the plurality of worn items based on the user state for each of the plurality of worn items; A recommendation unit that generates recommendation information indicating a wearable item that is recommended to the user to wear based on the evaluation information for each of the plurality of wearable items, The output unit further outputs the recommendation information. 10. The virtual try-on device of embodiment 9.

[0103] According to this aspect, recommended information for items to be worn is generated and output, so that information on items to be worn that are suitable for the user can be provided.

[0104] Aspect 11 The user information acquisition unit further acquires preference information regarding the preferences of the user, The evaluation unit generates the evaluation information further based on the preference information. 11. The virtual try-on device according to embodiment 9 or 10.

[0105] According to this aspect, evaluation information on the wearable item based on information regarding the user's preferences is generated and output, so that the accuracy of the evaluation information can be further improved.

[0106] Aspect 12 The user information acquisition unit further acquires feedback information input by the user after actually wearing another wearable item different from the predetermined wearable item and performing the predetermined exercise, The prediction unit predicts the user state further based on the feedback information. 12. The virtual try-on device according to any one of aspects 1 to 11.

[0107] According to this aspect, since the user state is predicted further based on the feedback information, the accuracy of the prediction can be further improved.

[0108] Aspect 13 A function to acquire the user's physical information; A function for acquiring information on a particular item of wear related to characteristics of the item of wear; A function of acquiring information regarding a situation when performing a predetermined exercise as a prediction condition; A function of predicting a user's condition when the user wears the predetermined item of wear and performs the predetermined exercise based on the body information, the item of wear information, and the prediction conditions; a function for outputting information regarding the user status; A virtual try-on program that makes this possible on a computer.

[0109] According to the present embodiment, the user's condition when wearing a specified item of wear and performing a specified exercise is predicted and output based on the user's physical information, information regarding the characteristics of the specified item of wear, and information regarding the circumstances when performing the specified exercise, so that the item of wear can be accurately evaluated from the outputted user condition. [Explanation of symbols]

[0110] 20...virtual try-on server, 46...output unit, 50...information acquisition unit, 52...user information acquisition unit, 54...wearing goods information acquisition unit, 56...prediction condition acquisition unit, 58...period acquisition unit, 60...reference action acquisition unit, 62...prediction unit, 64...evaluation unit, 66...recommendation unit, 70...avatar generation unit, 100...virtual try-on system.

Claims

1. A user information acquisition unit that acquires physical information of a user; A wearable item information acquisition unit that acquires wearable item information related to characteristics of a predetermined wearable item; a prediction condition acquisition unit that acquires information regarding a situation when performing a predetermined exercise as a prediction condition; a prediction unit that predicts a user's state when the user wears the predetermined item of wear and performs the predetermined exercise based on the body information, the item of wear information, and the prediction conditions; an output unit that outputs information regarding the user state; A virtual try-on device comprising:

2. an avatar generation unit that generates an avatar image in which the avatar modeled based on the physical information reflects the user's state; The output unit outputs the avatar image. The virtual try-on device according to claim 1 .

3. The user information acquisition unit further acquires exercise data when the user actually performs the predetermined exercise, A reference motion acquisition unit that acquires reference motion information that defines standard motions of a person performing the predetermined exercise, The prediction unit predicts a user's motion when the user wears the predetermined item of wear and performs the predetermined exercise based on the item of wear information, the exercise data, and the reference motion information; The avatar generation unit generates the avatar image so as to represent a situation in which the avatar performs the predetermined exercise through the user's action. The virtual try-on device according to claim 2 .

4. The prediction unit predicts, as the user state, a post-exercise user state after the user wears the predetermined wearable item and continues the predetermined exercise for a predetermined period of time. The virtual try-on device according to claim 3 .

5. the prediction unit predicts at least one of a change in the user's posture, a change in the user's movement, and a change in the specified item of wear caused by the user continuing the specified exercise for the specified period while wearing the specified item of wear, and predicts the user's state after exercise based on a prediction result; The virtual try-on device according to claim 4.

6. The virtual try-on device according to claim 4 or 5, wherein the post-exercise user state includes information on a performance index of the predetermined exercise.

7. The virtual try-on device according to claim 4 or 5, wherein the post-exercise user state includes information on the fit of the specified wearable item.

8. A period acquisition unit for acquiring period information is further provided, The virtual try-on device according to claim 4 , wherein the prediction unit predicts the post-exercise state of the user using the period acquired by the period acquisition unit as the predetermined period.

9. An evaluation unit that generates evaluation information of the predetermined wearing item based on the user state, The virtual try-on device according to claim 1 , wherein the output unit further outputs the evaluation information.

10. The wearing item information acquisition unit acquires the wearing item information for each of a plurality of wearing items, The prediction unit predicts the user state for each of the plurality of wearing items, The evaluation unit generates the evaluation information for each of the plurality of worn items based on the user state for each of the plurality of worn items; A recommendation unit that generates recommendation information indicating a wearable item that is recommended to the user to wear based on the evaluation information for each of the plurality of wearable items, The output unit further outputs the recommendation information. The virtual try-on device according to claim 9.

11. The user information acquisition unit further acquires preference information regarding the preferences of the user, The evaluation unit generates the evaluation information further based on the preference information. The virtual try-on device according to claim 9 or 10.

12. The user information acquisition unit further acquires feedback information input by the user after actually wearing another wearable item different from the predetermined wearable item and performing the predetermined exercise, The prediction unit predicts the user state further based on the feedback information. The virtual try-on device according to claim 1 .

13. A function to acquire the user's physical information; A function for acquiring information on a particular item of wear related to characteristics of the item of wear; A function of acquiring information regarding a situation when performing a predetermined exercise as a prediction condition; A function of predicting a user's condition when the user wears the predetermined item of wear and performs the predetermined exercise based on the body information, the item of wear information, and the prediction conditions; a function for outputting information regarding the user status; A virtual try-on program that makes this possible on a computer.

Images