Virtual try-on device and virtual try-on program product

Abstract

The invention provides a virtual try-on device and a virtual try-on program product capable of evaluating a wearing article with good precision. A virtual fitting device is provided with a user information acquisition unit (52), a wearing article information acquisition unit (54), a prediction condition acquisition unit (56), a prediction unit (62), and an output unit (46). A user information acquisition unit (52) acquires body information of a user. A wearing article information acquisition unit (54) acquires wearing article information relating to a characteristic of a predetermined wearing article. A prediction condition acquisition unit (56) acquires, as a prediction condition, information relating to a state when a predetermined motion is performed. The prediction unit (62) predicts the state of the user when the user wears a predetermined wearing article and performs a predetermined motion on the basis of the body information, the wearing article information, and the prediction conditions. The output unit (46) outputs information relating to the predicted user state.

Description

Technical Field

[0001] The present invention relates to a virtual try-on device and a virtual try-on program product. Background Art

[0002] A virtual fitting technology is known, which generates a composite image of a user wearing a wearable article such as clothes, so that the user can virtually experience fitting the wearable article without actually wearing the wearable article. For example, Patent Document 1 describes a technology that uses information about the user's movements to estimate the movement of the clothes when the user wears the clothes, and synthesizes the image of the clothes in the estimated movement into a moving image of the user. Thus, the technology described in Patent Document 1 presents the user with a dynamic image that is close to the state of the user wearing the actual clothes.

[0003] [Prior art literature]

[0004] [Patent Document]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 2006-249618 Summary of the invention

[0006] [Problems to be solved by the invention]

[0007] However, the technology described in Patent Document 1 does not take into account the conditions under which the wearer exercises when actually wearing the wearable article. Therefore, there is room for improvement in the evaluation of wearable articles.

[0008] The present disclosure has been made in view of such circumstances, and an object of the present disclosure is to provide a virtual try-on technology that can evaluate a wearable article with high accuracy.

[0009] [Technical means to solve the problem]

[0010] A virtual try-on device according to an embodiment of the present disclosure includes: a user information acquisition unit for acquiring body information of a user; a wear item information acquisition unit for acquiring wear item information related to characteristics of a specified wear item; a prediction condition acquisition unit for acquiring information related to a condition when performing a specified exercise as a prediction condition; a prediction unit for predicting a user state when the user wears the specified wear item and performs the specified exercise based on the body information, the wear item information and the prediction condition; and an output unit for outputting information related to the user state.

[0011] Another embodiment of the present disclosure is a virtual try-on program product. The program product enables a computer to implement the following functions: a function of acquiring a user's body information; a function of acquiring wear item information related to the characteristics of a specified wear item; a function of acquiring information related to a state when performing a specified exercise as a prediction condition; a function of predicting the user's state when the user wears the specified wear item and performs the specified exercise based on the body information, the wear item information, and the prediction condition; and a function of outputting information related to the user's state.

[0012] In addition, any combination of the above-mentioned constituent elements, or forms in which the constituent elements of the present invention or expressions in methods, apparatuses, programs, temporary or non-temporary storage media storing the programs, systems, etc. are mutually replaced are also valid forms of the present invention.

[0013] [Effects of the Invention]

[0014] According to an embodiment of the present disclosure, a virtual try-on technology capable of evaluating a wearable article with high accuracy can be provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram showing an overview of a virtual try-on system.

[0016] Figure 2 This is a functional block diagram showing a schematic structure of a virtual try-on system.

[0017] Figure 3 This is a functional block diagram showing a schematic configuration of a computing unit of a virtual try-on server.

[0018] Figure 4 This is a functional block diagram showing a schematic structure of a user information acquisition unit.

[0019] Figure 5 This is a functional block diagram showing a schematic structure of a prediction unit.

[0020] Figure 6 This is an example of the menu screen of the virtual try-on mode.

[0021] Figure 7 This is an example of a user information input screen.

[0022] Figure 8 This is an example of a screen for selecting a face image included in an avatar.

[0023] Fig. 9 This is an example of a selection screen for selecting a target wearable item.

[0024] Fig.10 This is an example of a display screen that displays an avatar image.

[0025] Fig.11 This is an example of a display screen that displays an avatar image.

[0026] [Explanation of Symbols]

[0027] 20: Virtual Fitting Server

[0028] 46: Output

[0029] 50: Information Acquisition Department

[0030] 52: User information acquisition department

[0031] 54: Wearable item information acquisition department

[0032] 56: Prediction condition acquisition department

[0033] 58: Period Acquisition Department

[0034] 60: Reference action acquisition unit

[0035] 62: Forecasting Department

[0036] 64: Evaluation Department

[0037] 66: Recommended

[0038] 70: Avatar Generation Department

[0039] 100: Virtual Try-On System DETAILED DESCRIPTION

[0040] In the embodiment and the modification examples, the same or equivalent components are denoted by the same reference numerals and duplicate descriptions are omitted as appropriate.

[0041] Figure 1 1 is a schematic diagram showing an overview of the virtual try-on system 100. The virtual try-on system 100 mainly includes a user terminal 10 and a virtual try-on server 20. The virtual try-on system 100 may also include a wearable item information server 2, and the function of the wearable item information server 2 may be constructed inside the virtual try-on server 20. Similarly, the virtual try-on system 100 may also include a reference motion information server 4, and the function of the reference motion information server 4 may be constructed inside the virtual try-on server 20. The components included in the virtual try-on system 100 are connected to each other via wireless communication 16 and a network 18.

[0042] The user terminal 10 and the virtual try-on server 20 may include: a portable terminal or computer including a central processing unit (CPU), a graphics processing unit (GPU), a random access memory (RAM), a read only memory (ROM), an auxiliary storage device, a display device, a communication device, etc., and a program stored in the portable terminal or computer. For example, the user terminal 10 may be configured in the form of a program executed by the virtual try-on server 20 via the network 18. It may also be configured so that the functions of the virtual try-on system are realized by a single device having the functions of the user terminal 10 and the virtual try-on server 20, and the virtual try-on program can be executed by the user directly operating the device. The single device may be a personal computer, a portable terminal such as a smartphone, an information terminal such as a tablet terminal, or a terminal operated by an assistant such as a clerk in a store selling wearable items, and a program stored in the terminal.

[0043] In this specification, the "virtual try-on device" may refer to the entire virtual try-on system 100, or may refer to the virtual try-on server 20. In this embodiment, since most of the characteristic functions included in the "virtual try-on device" are implemented in the form of including the virtual try-on server 20, the virtual try-on server 20 is substantially equivalent to the "virtual try-on device". However, the characteristic functions of the "virtual try-on device" may be dispersed in the user terminal 10 and the virtual try-on server 20, or the user terminal 10 may have most of them.

[0044] Reference Figure 1 , an example of the operation of the virtual try-on system 100 is described. The user uses the user terminal 10 to input the user's own body information, the specified information of the wearable article to be virtually tried on (hereinafter referred to as the "target wearable article"), and the information related to the state when performing the exercise assumed to be performed wearing the target wearable article (hereinafter referred to as the "target exercise"). The specified information of the target wearable article is, for example, the product number information of the target wearable article. The information related to the state when performing the target exercise can be used as a prediction condition for predicting the user state described later.

[0045] The user terminal 10 transmits the user's body information, the designated information of the target wearable article, and the prediction condition to the virtual fitting server 20 via the wireless communication 16 and the network 18. The virtual fitting server 20 obtains information related to the characteristics of the target wearable article (hereinafter referred to as "wearable article information") from the wearable article information server 2. The virtual fitting server 20 predicts the user's state when the user wears the target wearable article and performs the target exercise based on the acquired body information, wearable article information, and prediction condition. For example, when the target exercise is running, the user state includes information such as the breathability of the target wearable article and the burden on the body.

[0046] The virtual try-on server 20 models the avatar based on the user's body information. The virtual try-on server 20 generates an avatar image that reflects the predicted user state in the modeled avatar. The virtual try-on server 20 sends the generated avatar image to the user terminal 10. The user terminal 10 displays the received avatar image. In this way, the virtual try-on system 100 can reflect and display the user's state when the user wears a specified wearable item and performs a specified exercise in the avatar modeled based on the user's body information.

[0047] Figure 2 1 is a functional block diagram showing a schematic structure of the virtual try-on system 100. Figure 2 Each functional block shown in the figures may be implemented in hardware by an element or mechanical device represented by a computer processor or memory, and in software by a computer program, etc., but the functional blocks implemented by their cooperation are described here. These functional blocks may be implemented in various forms by hardware, software, or a combination thereof.

[0048] The user terminal 10 includes an operation processing unit 30, a display unit 32, a communication unit 34, and a storage unit 36. The operation processing unit 30 receives the user's operation. The operation processing unit 30, for example, receives the user's selection or input of the user's physical information, the designated information of the object to be worn, the prediction conditions, etc. The display unit 32 displays the information received from the virtual try-on server 20, such as the virtual avatar image, evaluation information, recommendation information, etc. described later. The operation processing unit 30, for example, receives an operation instruction for the information displayed on the display unit 32. The operation processing unit 30 and the display unit 32 can be integrally constituted by a touch panel in hardware, for example. The storage unit 36 ​​stores, for example, the user's physical information, the exercise data when the user actually performs a prescribed exercise including the object exercise, and information related to the user's preferences. The details of this information will be described later.

[0049] The communication unit 34 transmits the information selected or input by the operation processing unit 30 and the information stored in the storage unit 36 ​​to the virtual try-on server 20 via the network 18. The communication unit 34 receives information from the virtual try-on server 20 and transmits it to the display unit 32. The communication unit 34 may include a wireless communication module such as a wireless local area network (LAN) communication or a mobile phone communication in terms of hardware.

[0050] The virtual try-on server 20 includes a communication unit 40, a computing unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives information sent from the user terminal 10 and sends it to the computing unit 42, and sends information such as a computing result based on the computing unit 42 to the user terminal 10. The communication unit 40 may include a communication module such as a wired LAN in terms of hardware.

[0051] The calculation unit 42 performs calculation processing based on the information received by the communication unit 40. The calculation processing performed by the calculation unit 42 will be described in detail later. The calculation unit 42 stores the calculation result in the storage unit 44. The calculation result stored in the storage unit 44 is output from the output unit 46 and sent to the user terminal 10 via the communication unit 40.

[0052] The wearable article information server 2 stores wearable article information about one or more wearable articles including the target wearable article in a corresponding relationship with identification information such as identification (ID). The operation unit 42 of the virtual try-on server 20 requests the wearable article information server 2 to transmit the wearable article information of the target wearable article via the communication unit 40 based on the designated information of the target wearable article received from the user terminal 10. The wearable article information server 2 transmits the wearable article information of the target wearable article to the virtual try-on server 20 in response to the request from the virtual try-on server 20. The details of the wearable article information will be described later.

[0053] The reference motion information server 4 stores information that determines the standard motion of a person performing the motion for one or more motions including the object motion (hereinafter referred to as "reference motion information"). The calculation unit 42 of the virtual try-on server 20 requests the reference motion information server 4 to send the reference motion information of the object motion via the communication unit 40 based on the information related to the situation when the object motion is performed, that is, the prediction condition, received from the user terminal 10. The reference motion information server 4 sends the reference motion information of the object motion to the virtual try-on server 20 in response to the request from the virtual try-on server 20. The details of the reference motion information will be described later.

[0054] Figure 32 is a functional block diagram showing a schematic configuration of the computing unit 42 of the virtual try-on server 20. The computing unit 42 includes an information acquisition unit 50, a prediction unit 62, an avatar generation unit 70, an evaluation unit 64, and a recommendation unit 66.

[0055] The information acquisition unit 50 includes a user information acquisition unit 52, a worn article information acquisition unit 54, a prediction condition acquisition unit 56, a period acquisition unit 58, and a reference action acquisition unit 60. The user information acquisition unit 52 acquires information on the user of the user terminal 10.

[0056] Figure 4 2 is a functional block diagram showing a schematic configuration of the user information acquisition unit 52. The user information acquisition unit 52 of the present embodiment includes a body information acquisition unit 82, an exercise data acquisition unit 84, a preference information acquisition unit 86, and a feedback acquisition unit 88.

[0057] The body information acquisition unit 82 acquires the body information of the user. The body information of the user includes, for example, the height, weight, body fat percentage, fasting or postprandial blood sugar level, body temperature, and information on the size or shape of the body part of the object wearing the object. If the object wearing article is, for example, footwear or socks, the body information of the user may include foot shape information such as foot length or foot circumference. If the object wearing article is, for example, clothing, the body information of the user may include body shape information. The body information of the user may include information on the injury history of the body part of the object wearing the object. The body information acquisition unit 82 may read image information such as a photograph of the entire body or a specific body part of the user, and determine the body information of the user based on the image information.

[0058] The motion data acquisition unit 84 acquires motion data. The motion data acquired by the motion data acquisition unit 84 may be data recorded when the user actually performs the target motion, data estimated based on data recorded when the user performs a motion other than the target motion, or data estimated based on the user's physical information. The target motion may be any sport. In this specification, sports include not only running, ball games, swimming, cycling, skiing, skateboarding, and other competitions with relatively high loads, but also walking, stretching, and other sports with relatively low loads.

[0059] The motion data may include various measurement data obtained when the user actually performs the object motion. For example, the motion data may include data related to the action measured by a motion sensor such as a 9-axis sensor. The motion data may include biological data measured by a biosensor such as a heartbeat sensor or a body temperature sensor. The motion data may include data such as position coordinates or altitude measured by a position information sensor such as a positioning sensor or an altitude sensor. The motion data may include data such as temperature, air pressure, humidity, wind speed, etc. measured by an environmental sensor such as a temperature sensor or an air pressure sensor. The motion data may include weather information obtained via the Internet, etc. The motion data is not limited to data obtained by a single measurement, and may be data obtained by multiple measurements when the user performs a single object motion, or may be data obtained by measuring more than once when each object motion is performed when the user performs multiple object motions.

[0060] When the object movement is running or walking, the motion data may include, for example, stride length, interval (called the number of steps per unit time, also called cadence), intensity of ground impact, walking or walking distance, time required, heart rate, maximum oxygen uptake, posture-related analytical values, walking or walking path, elevation obtained, etc.

[0061] The motion data acquisition unit 84 may acquire the motion data from an existing application or an Internet service server that manages and records the motion data.

[0062] The preference information acquisition unit 86 acquires information related to the user's preferences. The information related to the user's preferences is information related to the user's preferences related to the target wearable article, and may include, for example, information on whether the user prefers a large or small size, information on whether the user prefers a hard or soft ground contact when the target wearable article is a shoe, information on the favorite color, information on the favorite design, etc. The information related to the user's preferences may include information on the wearable article that the user has worn in the past and information on the user's feelings after wearing the wearable article. The user's feelings may include information on the fit, information indicating the parts of the wearable article that are easily worn, etc. The information related to the user's preferences may include information on the direction of the performance required by the user for the target wearable article, such as whether the user attaches importance to functionality in the target sport or fashion. The information related to the user's preferences may be information input by the user using the user terminal 10, or may be information generated based on the user's past purchase history or search history of wearable articles, or may be information generated through a perceptual survey, a questionnaire survey, etc. The information related to the user's preferences may be information inferred based on information related to actions such as postings and activities on social media, or information related to the content of the account being followed.

[0063] The feedback acquisition unit 88 acquires feedback information input by a user who actually wears another wearable article different from the target wearable article and performs the target exercise. The details will be described later. The feedback information can be used by the prediction unit 62 to predict the user state and can be effectively used to improve the prediction accuracy.

[0064] return Figure 3 , the wear item information acquisition unit 54 acquires wear item information related to the characteristics of the object wear item. The object wear item is, for example, an item used for sports. The object wear item includes footwear, insoles of footwear, socks, bandages, compression clothing, clothes, decorative tools, protective gear, etc. The wear item information may include various information such as the shape of the object wear item, the material of each part, the quality, the compression force of the wearing part, the color, the size, and other information that may affect the performance of the object's movement. The wear item information acquisition unit 54 may acquire, from the wear item information server 2, the wear item information corresponding to the specified information (e.g., the product number) of the object wear item received from the user terminal 10.

[0065] The prediction condition acquisition unit 56 acquires information related to the situation when the target sport is performed as a prediction condition. The prediction condition can be used as a condition when the prediction unit 62 predicts the user state. The prediction condition may include information related to the environment when the target sport is performed, information related to the content of the sport, etc. For example, when the target sport is running, the prediction condition may include information about the track, such as whether it is a paved road or an unpaved road, information about the climate, information about the wind speed, information about the walking distance, information about the walking speed, etc. For example, when the target sport is football, the prediction condition may include information about the surface type of the football field, such as whether it is natural grass, artificial grass, or soil.

[0066] The period acquisition unit 58 acquires information about the period. The details will be described later, and the information about the period acquired by the period acquisition unit 58 is used by the prediction unit 62 to generate the post-wearing information. That is, the information about the period acquired by the period acquisition unit 58 is used as information about the prescribed period when the prediction unit 62 predicts the post-exercise user state after the user wears the object wearing article and continues the object exercise for a prescribed period.

[0067] The reference motion acquisition unit 60 acquires reference motion information. The reference motion information is information about a motion model determined by the type of the object motion. For example, when the object motion is running, it may be information about a human body model walking in a standard posture along the time axis. The details will be described later. The reference motion information is used as a reference motion when the prediction unit 62 predicts the user motion. The reference motion acquisition unit 60 may acquire, from the reference motion information server 4, the reference motion information corresponding to the specified information of the object motion received from the user terminal 10.

[0068] The prediction unit 62 predicts the user state when the user wears the target worn article and performs the target exercise based on the body information acquired by the body information acquisition unit 82 , the worn article information acquired by the worn article information acquisition unit 54 , and the prediction condition acquired by the prediction condition acquisition unit 56 .

[0069] The user state predicted by the prediction unit 62 may include information related to various effects obtained by the user by wearing the target wearable article and performing the target exercise. The user state may include, for example, information related to the physical properties of the target wearable article, such as the breathability of the target wearable article, the pressure applied to the wearing part, the temperature distribution, the burden on the body, the pressure on the joints, the fatigue of the muscles, the amount of sweating, or information related to the user's body. The user state is information that can be used to evaluate the target wearable article.

[0070] Figure 5 2 is a functional block diagram showing a schematic configuration of the prediction unit 62. The prediction unit 62 of the present embodiment includes a user motion prediction unit 90, a posture change prediction unit 92, a motion change prediction unit 94, and a worn article change prediction unit 96.

[0071] The user action prediction unit 90 predicts the user action when the user wears the target wearable article and performs the target exercise based on the wearable article information of the target wearable article acquired by the wearable article information acquisition unit 54, the exercise data acquired by the exercise data acquisition unit 84, and the reference exercise information acquired by the reference exercise acquisition unit 60. The user action predicted by the user action prediction unit 90 is an example of the user state predicted by the prediction unit 62.

[0072] Specifically, first, the user action prediction unit 90 predicts the action of the user when performing the target motion based on the difference between the reference motion information and the motion data. Here, the motion data can be used as a parameter that reflects the user's habits or characteristics in the standard action based on the reference motion information and is used to predict the user's inherent action. Next, the user action prediction unit 90 makes adjustments based on the wearing article information of the target wearable article so that the predicted user action becomes an action when the target wearable article is worn. For example, in the case where the target wearable article is a shoe, when the mass of the shoe is greater than that of ordinary shoes, the spacing as the user action may be reduced, or the posture may change. The degree of change of the user action based on the wearing article information of the target wearable article may vary depending on the motion data. For example, the weaker the user's muscle strength, the more easily the user action may be affected by the mass of the target wearable article.

[0073] The posture change prediction unit 92 predicts changes in the user's posture caused by the user wearing the target wearable article and continuing the target movement for a predetermined period. The motion change prediction unit 94 predicts changes in the user's motion caused by the user wearing the target wearable article and continuing the target movement for a predetermined period. The wearable article change prediction unit 96 predicts changes in the target wearable article caused by the user wearing the target wearable article and continuing the target movement for a predetermined period.

[0074] In the present specification, the prescribed period may be a period acquired by the period acquisition unit 58, or a pre-set period. The pre-set period may be information of a period that has been pre-established with at least one of the object's wearable article and the object's movement. The prescribed period may vary according to the object's movement, and may include, for example, a short period such as a single movement (a game, a competition, etc.) to a long period such as several years. The "object movement that continues for a prescribed period" is not limited to continuing the object movement without rest, and includes habitually performing the object movement. In addition, the prescribed period is not limited to a certain period such as 1 hour, and may also be a period such as a period until a game is established, a period until a race is completed, and a period until a certain condition is met.

[0075] The posture change prediction unit 92 predicts a change in the user's posture based on the reference motion information acquired by the reference motion acquisition unit 60, the motion data acquired by the motion data acquisition unit 84, and the wearable article information of the target wearable article acquired by the wearable article information acquisition unit 54. For example, the posture change prediction unit 92 predicts to what extent the user's posture is deformed due to the user wearing the target wearable article for a predetermined period and continuing the target motion based on the reference motion information and the motion data. For example, the posture change prediction unit 92 predicts the influence of a change in the support function of the target wearable article on a change in the user's posture after the user wears the target wearable article and continues the target motion for a predetermined period based on the reference motion information, the motion data, and the wearable article information of the target wearable article.

[0076] Multiple parameters related to the motion data and multiple parameters related to the wearable article information of the object wearable article can change individually or while interacting with each other according to the duration of the object motion. For example, when the object motion is running or walking, the motion data includes a parameter of stride and a parameter of spacing. The parameter of stride and the parameter of spacing tend to decrease as the duration of the object motion increases. The user's motion data also includes information on individual differences in the degree of temporal change of each parameter. That is, the degree of temporal change of the parameter of stride and the degree of temporal change of the parameter of spacing differ from user to user. The wearable article information of the object wearable article includes a parameter of mass. The parameter of mass is constant regardless of the duration of the object motion. However, the larger the parameter of mass, the easier it is for the user to fatigue, so the parameter of stride and the parameter of spacing as parameters of the motion data can be reduced. In this way, the posture change prediction unit 92 can use the information of the temporal change of each parameter to predict the change of the user's posture. The same is true for the prediction of the change of the user's motion by the motion change prediction unit 94 and the prediction of the change of the object wearable article by the wearable article change prediction unit 96.

[0077] For example, when the target sport is running, the change in the user's posture is a change in the running posture, and the posture change prediction unit 92 predicts, for example, how much the waist will drop or how much the upper body will lean forward in the final stage of a long-distance race such as a marathon. The posture change prediction unit 92 may also predict a change in the shape of the part where the target wearable article is worn, for example, if the target wearable article is a shoe, predicting swelling of the foot, etc. as a change in the user's posture. The posture change prediction unit 92 may predict changes in the user's expression, etc., caused by the user wearing the target wearable article and continuing the target sport for a predetermined period of time based on the reference motion information, the motion data, and the wearable article information of the target wearable article.

[0078] The motion change prediction unit 94 predicts the change of the user's motion based on the reference motion information acquired by the reference motion acquisition unit 60, the motion data acquired by the motion data acquisition unit 84, and the wear article information of the target wear article acquired by the wear article information acquisition unit 54. For example, the motion change prediction unit 94 predicts the change of the user's motion caused by the fatigue of the user caused by the user wearing the target wear article for a relatively short period of time and continuously performing the target motion based on the reference motion information and the motion data. For example, the motion change prediction unit 94 predicts the influence of the user's habit or familiarity with the target wear article on the user's motion after the user wears the target wear article for a relatively long period of time and continuously performs the target motion based on the reference motion information, the motion data, and the wear article information of the target wear article.

[0079] The wear article change prediction unit 96 predicts a change in the target wear article based on the reference motion information acquired by the reference motion acquisition unit 60, the motion data acquired by the motion data acquisition unit 84, and the wear article information of the target wear article acquired by the wear article information acquisition unit 54. For example, the wear article change prediction unit 96 predicts a change in the elasticity of the target wear article, a change in the supporting force of the contact part with the user's body, and the like, caused by the user wearing the target wear article for a predetermined period and continuously performing the target motion, based on the reference motion information, the motion data, and the wear article information of the target wear article. When it is considered that the target wear article will not undergo a large change in a short period, the wear article change prediction unit 96 may predict that the target wear article will not undergo a change if the predetermined period is a period below a certain threshold.

[0080] As described above, the posture change prediction unit 92, the motion change prediction unit 94, and the wear article change prediction unit 96 respectively predict the change in the user's posture, the change in the user's motion, and the change in the object wear article caused by the user wearing the object wear article and continuing the object motion for a specified period. The prediction unit 62 predicts the post-exercise user state after the user wears the object wear article and continues the object motion for a specified period based on the prediction result of at least any one of the change in the user's posture, the change in the user's motion, and the change in the object wear article caused by the user wearing the object wear article and continuing the object motion for a specified period. The post-exercise user state is an example of the user state. The post-exercise user state may also be the same as the information of the prediction result. The prediction unit 62 may also predict the staged changes in the user state when the user continues the object motion for a specified period as the post-exercise user state. For example, when the object motion is running, when the specified period is a marathon, the prediction unit 62 may predict the changes in the user state every 5 km the user walks as the post-exercise user state.

[0081] The user status after exercise may include information on the performance index of the target exercise. For example, when the target exercise is running, the information on the performance index may be an analytical value related to the running posture, a predicted value of the race time, etc. The information on the performance index is not limited to an absolute value, but may also be a relative value indicating the degree to which the performance index has changed in comparison with other wearable articles.

[0082] The user status after exercise may include a fit indicator indicating how well the object being worn fits the user.

[0083] The prediction unit 62 can further predict the user state based on the feedback information acquired by the feedback acquisition unit 88. As described above, the feedback information is information related to feedback input by the user who actually wears other wearable articles different from the target wearable article and performs the target exercise. In this way, the prediction unit 62 further predicts the user state based on the feedback information, whereby the feedback information supplements the user's physical information, thereby improving the accuracy of the prediction.

[0084] The user motion prediction unit 90 may correct the predicted user motion based on at least one of the prediction results of the change in the user's posture predicted by the posture change prediction unit 92, the change in the user's motion predicted by the motion change prediction unit 94, and the change in the target worn article predicted by the worn article change prediction unit 96. Thus, the user motion prediction unit 90 may predict the user motion after the user wears the target worn article and continues the target motion for a predetermined period of time.

[0085] The prediction made by the prediction unit 62 may use a prediction model learned by machine learning. In this case, when body information, wearable article information, prediction conditions, motion data, reference motion information, information during the period, and feedback information are input, the prediction model outputs a user state including user actions and user states after exercise. In addition, the prediction made by the prediction unit 62 may also use analytical techniques such as regression analysis or multivariate analysis. In this case, body information, wearable article information, prediction conditions, motion data, reference motion information, information during the period, and feedback information may be set as explanatory variables, and the user state including user actions and user states after exercise may be set as target variables.

[0086] return Figure 3 The virtual image generation unit 70 includes a modeling unit 72 and an image generation unit 74. The modeling unit 72 models the virtual image based on the user's body information. The virtual image is a character that is represented in the virtual space as the user's avatar, and can be modeled using well-known techniques. The virtual image can represent the user's entire body, or a specific body part of the user, such as the body part of the wearer's wearable item, or both. The virtual image can represent the user realistically, or can be represented by emphasizing the user's characteristics, or can be represented in a simplified manner using stick figures, or can be represented using characters such as animals. The modeling unit 72 can model a part of the virtual image, such as the type of face, based on the information selected by the user.

[0087] The modeling unit 72 reflects the user's body information in the avatar. For example, when the avatar represents the user's entire body, the modeling unit 72 models the avatar of a body type corresponding to the user's height, weight, body fat percentage, etc. For example, when the avatar represents a specific body part of the user, the avatar is modeled corresponding to the size of the body part of the user.

[0088] The modeling unit 72 may model the avatar based on the object wearing item information in addition to the user's body information. In this case, the modeling unit 72 models the avatar in a state where the object wearing item is worn. The avatar may change according to the type, size, shape, color, etc. of the object wearing item.

[0089] The image generation unit 74 generates an avatar image that reflects the user state predicted by the prediction unit 62 in the avatar modeled by the modeling unit 72. For example, when the user state is information related to the physical properties of the target wearable article, such as breathability, pressure, temperature distribution, etc., an avatar image is generated in which the display form such as the color distribution of the body part wearing the target wearable article is changed. For example, when the user state is information related to the user's body, such as muscle fatigue, sweating, etc., an avatar image is generated in which the display form such as the color distribution of the whole body or body parts, or the enlarged display of the target part is changed.

[0090] When the user state predicted by the prediction unit 62 is a post-exercise user state, the image generation unit 74 generates an avatar image reflecting the post-exercise user state. The avatar image reflecting the post-exercise user state may be in the same display form as the avatar image reflecting the user state. When the post-exercise user state predicted by the prediction unit 62 includes a plurality of staged user state changes, the image generation unit 74 may generate an avatar image corresponding to each change in the user state.

[0091] When the user motion prediction unit 90 predicts the user motion, the image generation unit 74 generates an avatar image to represent the avatar performing the target motion with the user motion. Thus, the image generation unit 74 can reflect information about how the user moves when wearing the target wearable article and performing the target motion in the avatar image.

[0092] The avatar image may be still image data, dynamic image data, or data modeled in three dimensions (3D). When the avatar image is 3D modeled data, after the avatar image is output by the output unit 46, when display control is performed using the user terminal 10, etc., the avatar image may be converted into a data format that can be displayed. The image generation unit 74 may set an image of the avatar observed from the perspective of a third party as the avatar image, or may set an image from the perspective of the avatar as the avatar image. When the avatar image is an image from the perspective of the avatar, for example, it is set as an image of a body part of a wearer's wearable article, thereby making it easy to imagine the user's state when the user actually wears the wearable article.

[0093] The evaluation unit 64 generates evaluation information of the target wearable article based on the user state predicted by the prediction unit 62. The evaluation unit 64 can evaluate the information contained in the information of the user state, such as information related to the physical properties of the target wearable article or information related to the user's body, and generate evaluation information including evaluation of each item and comprehensive evaluation. The evaluation information can include, for example, information on the score or grade corresponding to the comprehensive evaluation, and information on the score distribution corresponding to the evaluation of each item. The evaluation information can be displayed in the form of a chart or a numerical value, and can also be reflected in the avatar image by the image generation unit 74, for example, the expression of the avatar image can be changed according to the comprehensive evaluation.

[0094] The recommendation unit 66 generates recommendation information indicating the wearable article recommended to the user based on the wearable article information for each of the plurality of wearable articles. In this case, the virtual try-on server 20 acquires the wearable article information for each of the plurality of wearable articles by, for example, the following processing. The wearable article information acquisition unit 54 acquires the wearable article information for each of the plurality of wearable articles from the wearable article information server 2. Here, the wearable article information acquired by the wearable article information acquisition unit 54 may be the wearable article information designated as the target wearable article, or may be the wearable article information that satisfies the prescribed condition when not designated as the target wearable article. The prediction unit 62 predicts the user status for each of the plurality of wearable articles. The evaluation unit 64 generates evaluation information for each of the plurality of wearable articles based on the user status for each of the plurality of wearable articles. Based on the evaluation information for each of the plurality of wearable articles thus obtained, the recommendation unit 66 may generate, for example, the information of the wearable article having the highest comprehensive evaluation of the evaluation information as the recommendation information recommended to the user. In addition, the image generation unit 74 may generate, respectively, the avatar images reflecting the user status for each of the plurality of wearable articles predicted by the prediction unit 62. Thus, it is easy to display and compare the avatar images for the plurality of wearable articles, for example, simultaneously.

[0095] return Figure 2 The storage unit 44 stores the calculation results of the avatar image and the like generated by the calculation unit 42. The storage unit 44 may also store in advance information used by the calculation unit 42 for calculation.

[0096] The output unit 46 outputs the avatar image generated by the avatar generation unit 70. The output unit 46 may output the information related to the user status predicted by the prediction unit 62 in a form other than the avatar image, such as text, numerical value, and chart information. Among them, by outputting in the form of a avatar image, the user can intuitively grasp the user status, which is preferable in this respect. The output unit 46 may also output the evaluation information generated by the evaluation unit 64. The output unit 46 may also output the recommendation information generated by the recommendation unit 66. The information output by the output unit 46 is sent to the user terminal 10 via the communication unit 40.

[0097] Figure 6 to Figure 11 This is an example of a display screen displayed on the display unit 32 of the user terminal 10 . Figure 6 This is an example of the menu screen for the virtual try-on mode. Figure 6 When the "virtual image creation" image 112 is displayed, the display unit 32 transitions to Figure 7 display screen.

[0098] Figure 7 This is an example of a user information input screen. Figure 7 When the "Physical Information Input" image 124 is displayed, the user terminal 10 accepts input of physical information such as body shape or height. Figure 7 When the "Measurement Information" image 122 is displayed, the virtual try-on server 20 obtains the user's exercise data from, for example, a server that manages a third-party service. Figure 7 When the "generate" image 126 is displayed, the user terminal 10 sends the input body information to the virtual try-on server 20, and the virtual try-on server 20 generates a virtual image based on the body information and motion data.

[0099] Figure 8 This is an example of a screen for selecting a face image included in an avatar. Figure 8 As shown, the user can change the face image included in the avatar.

[0100] When choosing Figure 6 When the "Select Shoes" image 114 is displayed, the display unit 32 transitions to Fig. 9 display screen. Fig. 9 This is an example of a selection screen for selecting a target wearable item. Fig. 9 In the virtual try-on server 20, one or more shoes are displayed so that one or more shoes can be selected as the target wearable articles. The user terminal 10 sends information designating the selected shoes as the target wearable articles to the virtual try-on server 20.

[0101] exist Figure 6 The user terminal 10 displays images corresponding to a plurality of types of sports below the "virtual try-on" image 116. For example, when the running image 118 is selected, the user terminal 10 sends information specifying the target sport as running to the virtual try-on server 20. The user terminal 10 further receives input of information related to the condition when running is performed as the target sport, and sends it to the virtual try-on server 20 as a prediction condition.

[0102] Regarding the virtual try-on server 20, the prediction unit 62 predicts the user state when the user wears the shoes designated as the target worn items and performs running designated as the target exercise based on the input body information and the worn article information and the prediction condition. In addition, regarding the virtual try-on server 20, the avatar generation unit 70 reflects the predicted user state in the avatar modeled based on the body information and generates an avatar image.

[0103] Fig.10 and Fig.11 This is an example of a display screen showing the generated avatar image. Fig.10 The avatar image of the object wearing the item as Fig.11 Compared with the object worn by the virtual avatar image, when the user performs the object movement under the predicted conditions, it is shown that the user is less likely to get tired, the posture is easy to stabilize, and the wearing feeling is comfortable.

[0104] Here, a specific example of using the virtual try-on system 100 of the present embodiment is described. First, the case where the target sport is soccer is described. In this case, the user's body information includes information on height, weight, and foot shape. The target wearable item is footwear, and the wearable item information includes information on the material and structure of the upper and outsole. The prediction condition includes information on the surface type of the soccer field, such as whether it is natural grass, artificial grass, or soil. In addition, in this example, motion data may not be used.

[0105] The prediction unit 62 of this example predicts the information of the pressure applied to the foot and the shoe during the kicking or stepping action as the user state. The avatar generation unit 70 of this example models at least one of the whole body and the foot as an avatar, and generates an avatar image that displays the information of the pressure predicted by the prediction unit 62 by means of a color change or the like. In this way, the magnitude of the burden imposed on the foot by the object's wearing article can be predicted.

[0106] Next, the case where the object sport is running is described. In this case, the user's body information includes information about height, weight, and foot shape. The sports data includes information about the environment and movements in which the sport is performed. The object wears items such as footwear or running clothes. If they are footwear, the wear item information includes information about size, material, and structure. The material includes information about the material of the upper and the material of the sole. The structure includes information about the structure of the last, midsole, shoelaces, and upper. If it is sportswear, the wear item information includes information about the size, cutting pattern, and material of the fabric. The prediction conditions include information about the type of walking, temperature, humidity, season, wind speed, walking distance, walking speed, type of walking method, fasting status, and the like.

[0107] The prediction unit 62 of this example predicts the fatigue of muscles, the burden on the body, and the air permeability of running clothes as the user state. In addition, when the prediction unit 62 selects both shoes and running clothes as the target wearable items, the influence of their interaction is also included in the user state. The avatar generation unit 70 of this example models at least one of the whole body and the foot as an avatar, and generates an avatar image that displays the information predicted by the prediction unit 62 in the form of color changes or changes in the size of body parts. In this way, it is possible to predict the wearing feeling when using the target wearable items for a medium to long period of time.

[0108] As described above, according to the present embodiment, the prediction unit 62 predicts the user state when the user wears the target wearable article and performs the target exercise based on the body information, the wearable article information, and the prediction condition. The avatar generation unit 70 generates an avatar image that reflects the user state in the avatar modeled based on the body information. Thus, an avatar image that reflects the user state predicted based on the situation when performing a predetermined exercise can be generated. Therefore, by observing the avatar image, the user can intuitively and accurately evaluate the wearable article.

[0109] In addition, according to the present embodiment, the prediction unit 62 predicts the user's motion when the user wears the target wearable article and performs the target motion based on the wearable article information, the motion data, and the reference motion information. The avatar generation unit 70 generates an avatar image so as to represent the state when the avatar performs the target motion with the user's motion. Thus, an avatar image representing the user's motion reflecting the characteristics of the wearable article can be generated, so that the user can understand how the wearable article affects the motion.

[0110] In addition, according to the present embodiment, the prediction unit 62 predicts the user state after the user wears the target wearable article and continues the target exercise for a predetermined period as the user state. Specifically, the prediction unit 62 predicts at least one of the change in the user's posture, the change in the user's movement, and the change in the target wearable article caused by the user wearing the target wearable article and continuing the target exercise for a predetermined period, and predicts the user state after the exercise based on the prediction result. In this way, the state after the wearable article is used for a certain period of time can be predicted, so the wearable article can be evaluated with better accuracy.

[0111] The implementation method may be a program for causing a computer to implement the functions for implementing the method, or a recording medium storing the program. Such a recording medium storing the program may be a non-transitory and tangible computer-readable recording medium, or a non-volatile memory, a magnetic recording medium such as a tape or a disk, or an optical recording medium such as an optical disk.

[0112] The above is an explanation of the embodiments. The embodiments are for illustration only, and it is understood by those skilled in the art that the combination of these components or processing techniques can have various variations, and such variations are also within the scope of the present invention. In addition, if the embodiments are generalized, the following embodiments can be obtained.

[0113] [Implementation method 1]

[0114] A virtual try-on device, comprising:

[0115] A user information acquisition unit, which acquires the user's physical information;

[0116] a worn article information acquiring unit that acquires worn article information related to characteristics of a predetermined worn article;

[0117] A prediction condition acquisition unit that acquires information related to a situation when performing a predetermined exercise as a prediction condition;

[0118] a prediction unit that predicts a user state when the user wears the prescribed worn article and performs the prescribed exercise based on the body information, the worn article information, and the prediction condition; and

[0119] The output unit outputs information related to the user status.

[0120] According to this embodiment, based on the user's body information, information related to the characteristics of the specified wearable article, and information related to the situation when performing the specified exercise, the user's state when the user wears the specified wearable article and performs the specified exercise is predicted and output, so the wearable article can be evaluated with high accuracy based on the output user state.

[0121] [Implementation method 2]

[0122] According to the virtual try-on device described in embodiment 1,

[0123] The system further includes an avatar generating unit configured to generate an avatar image reflecting the user's state in an avatar modeled based on the body information.

[0124] The output unit outputs the avatar image.

[0125] According to the present embodiment, an avatar image reflecting the user's state is generated and output in an avatar based on the user's physical information, so that the user's state can be easily grasped intuitively from the output avatar image.

[0126] [Implementation method 3]

[0127] According to the virtual try-on device described in embodiment 2,

[0128] The user information acquisition unit further acquires the exercise data when the user actually performs the prescribed exercise.

[0129] The virtual try-on device further includes a reference motion acquisition unit configured to acquire reference motion information that determines a standard motion of a person performing the prescribed exercise.

[0130] the prediction unit predicts a user motion when the user wears the predetermined worn article and performs the predetermined motion based on the worn article information, the motion data, and the reference motion information,

[0131] The avatar generation unit generates the avatar image so as to represent a situation when the avatar performs the predetermined motion by the user action.

[0132] According to this embodiment, the situation when the user wears a predetermined wearing article and performs a predetermined exercise can be reflected in the avatar image.

[0133] [Implementation method 4]

[0134] According to the virtual try-on device described in embodiment 3,

[0135] The prediction unit predicts, as the user state, a post-exercise user state after the user wears the predetermined wearing article and continues the predetermined exercise for a predetermined period of time.

[0136] According to the present embodiment, the state of the worn article after it has been used for a certain period of time can be predicted, and thus the worn article can be evaluated with higher accuracy.

[0137] [Implementation method 5]

[0138] According to the virtual try-on device described in embodiment 4,

[0139] The prediction unit predicts at least any one of a change in the user's posture, a change in the user's movement, and a change in the prescribed wearable article caused by the user wearing the prescribed wearable article and continuing the prescribed exercise for the prescribed period, and predicts the user state after the exercise based on the prediction result.

[0140] According to this embodiment, the state of the worn article after a certain period of use can be predicted based on the prediction result of at least any one of the change in user posture, the change in user motion, and the change in the worn article, so the worn article can be evaluated with higher accuracy.

[0141] [Implementation method 6]

[0142] According to the virtual try-on device described in embodiment 4 or 5, the post-exercise user status includes information on performance indicators of the prescribed exercise.

[0143] According to the present embodiment, since information on the performance index of exercise after the worn article has been used for a certain period of time can be predicted, the worn article can be evaluated with higher accuracy.

[0144] [Implementation method 7]

[0145] A virtual try-on device according to any one of Embodiments 4 to 6, wherein the post-exercise user state includes information on a fit of the predetermined worn article.

[0146] According to the present embodiment, since information on the feeling of fit of a worn article after the worn article has been used for a certain period of time can be predicted, the worn article can be evaluated with higher accuracy.

[0147] [Implementation method 8]

[0148] The virtual try-on device according to any one of embodiments 4 to 7 further includes a period acquisition unit for acquiring information of the period.

[0149] The prediction unit predicts the user state after exercise by setting the period acquired by the period acquisition unit as the predetermined period.

[0150] According to the present embodiment, the state of the worn article after the wearable article has been used for a specified period can be predicted, and thus the worn article can be evaluated with higher accuracy.

[0151] [Implementation method 9]

[0152] The virtual try-on device according to any one of Embodiments 1 to 8 further includes an evaluation unit that generates evaluation information of the predetermined wearing article based on the user state.

[0153] The output unit further outputs the evaluation information.

[0154] According to the present embodiment, since evaluation information of the worn article is generated and output, it is possible to evaluate the worn article more objectively.

[0155] [Implementation method 10]

[0156] According to the virtual try-on device described in embodiment 9,

[0157] The worn article information acquiring unit acquires the worn article information of a plurality of worn articles respectively.

[0158] The prediction unit predicts the user status for each of the plurality of worn articles.

[0159] the evaluation unit generates the evaluation information for each of the plurality of worn articles based on the user status for each of the plurality of worn articles,

[0160] The virtual try-on device further includes a recommendation unit configured to generate recommendation information indicating a wearable article recommended to be worn by the user based on the evaluation information on each of the plurality of wearable articles.

[0161] The output unit further outputs the recommendation information.

[0162] According to the present embodiment, since recommendation information of worn articles is generated and output, it is possible to provide information of worn articles suitable for the user.

[0163] [Implementation method 11]

[0164] According to the virtual try-on device described in embodiment 9 or 10,

[0165] The user information acquisition unit further acquires preference information related to the user's preference,

[0166] The evaluation unit further generates the evaluation information based on the preference information.

[0167] According to the present embodiment, since evaluation information of a worn article based on information related to a user's preference is generated and output, the accuracy of the evaluation information can be further improved.

[0168] [Implementation method 12]

[0169] A virtual try-on device according to any one of embodiments 1 to 11, wherein:

[0170] The user information acquisition unit further acquires feedback information input by the user who actually wears another wearing article different from the predetermined wearing article and performs the predetermined exercise.

[0171] The prediction unit further predicts the user state based on the feedback information.

[0172] According to this embodiment, the user status is further predicted based on the feedback information, so the accuracy of the prediction can be further improved.

[0173] [Implementation method 13]

[0174] A virtual try-on program is used to enable a computer to achieve the following functions:

[0175] Function to obtain user's physical information;

[0176] A function for obtaining information about a worn article related to characteristics of a specified worn article;

[0177] A function of acquiring information related to the situation when performing a prescribed exercise as a prediction condition;

[0178] A function of predicting a user state when the user wears the prescribed worn article and performs the prescribed exercise based on the body information, the worn article information, and the prediction condition; and

[0179] A function of outputting information related to the user status.

[0180] According to this embodiment, based on the user's body information, information related to the characteristics of the specified wearable article, and information related to the situation when performing the specified exercise, the user's state when the user wears the specified wearable article and performs the specified exercise is predicted and output, so the wearable article can be evaluated with high accuracy based on the output user state.

Claims

1. A virtual try-on device, comprising: A user information acquisition unit, which acquires the user's physical information; a worn article information acquiring unit that acquires worn article information related to characteristics of a predetermined worn article; A prediction condition acquisition unit that acquires information related to a situation when performing a predetermined exercise as a prediction condition; a prediction unit that predicts a user state when the user wears the prescribed worn article and performs the prescribed exercise based on the body information, the worn article information, and the prediction condition; as well as The output unit outputs information related to the user status.

2. The virtual try-on device according to claim 1, The system further includes an avatar generating unit configured to generate an avatar image reflecting the user's state in an avatar modeled based on the body information. The output unit outputs the avatar image.

3. The virtual try-on device according to claim 2, wherein: The user information acquisition unit further acquires the exercise data when the user actually performs the prescribed exercise. The virtual try-on device further includes a reference motion acquisition unit configured to acquire reference motion information that determines a standard motion of a person performing the prescribed exercise. the prediction unit predicts a user motion when the user wears the predetermined worn article and performs the predetermined motion based on the worn article information, the motion data, and the reference motion information, The avatar generation unit generates the avatar image so as to represent a situation when the avatar performs the predetermined motion by the user action.

4. The virtual try-on device according to claim 3, wherein: The prediction unit predicts, as the user state, a post-exercise user state after the user wears the predetermined worn article and continues the predetermined exercise for a predetermined period of time.

5. The virtual try-on device according to claim 4, wherein: The prediction unit predicts at least any one of a change in the user's posture, a change in the user's movement, and a change in the prescribed wearable article caused by the user wearing the prescribed wearable article and continuing the prescribed exercise for the prescribed period, and predicts the user state after the exercise based on the prediction result.

6. The virtual try-on device according to claim 4 or 5, wherein: The post-exercise user status includes information on performance indicators of the prescribed exercise.

7. The virtual try-on device according to claim 4 or 5, wherein: The post-exercise user state includes information on the fit of the predetermined worn article.

8. The virtual try-on device according to claim 4, further comprising a period acquisition unit for acquiring information of the period, The prediction unit predicts the user state after exercise by setting the period acquired by the period acquisition unit as the predetermined period.

9. The virtual try-on device according to claim 1, further comprising an evaluation unit configured to generate evaluation information of the predetermined wearing article based on the user state. The output unit further outputs the evaluation information.

10. The virtual try-on device according to claim 9, wherein: The worn article information acquisition unit acquires the worn article information for each of a plurality of worn articles, The prediction unit predicts the user status for each of the plurality of worn articles. the evaluation unit generates the evaluation information for each of the plurality of worn articles based on the user status for each of the plurality of worn articles, The virtual try-on device further includes a recommendation unit configured to generate recommendation information indicating a wearable article recommended to be worn by the user based on the evaluation information on each of the plurality of wearable articles. The output unit further outputs the recommendation information.

11. The virtual try-on device according to claim 9 or 10, wherein: The user information acquisition unit further acquires preference information related to the user's preference, The evaluation unit further generates the evaluation information based on the preference information.

12. The virtual try-on device according to claim 1, wherein: The user information acquisition unit further acquires feedback information input by the user who actually wears another wearing article different from the predetermined wearing article and performs the predetermined exercise. The prediction unit further predicts the user state based on the feedback information.

13. A virtual try-on program product, used to enable a computer to implement the following functions: Function to obtain user's physical information; A function for obtaining information about a worn article related to characteristics of a specified worn article; A function of acquiring information related to the situation when performing a prescribed exercise as a prediction condition; A function of predicting a user state when the user wears the prescribed worn article and performs the prescribed exercise based on the body information, the worn article information, and the prediction condition; and A function of outputting information related to the user status.

Images