VIRTUAL TRY-ON DEVICE AND VIRTUAL TRY-ON PROGRAM

Abstract

A virtual fitting device (100), comprising: a user information capture unit (52) that captures physical information of a user; a garment information capture unit (54) that captures information about the garment in relation to the characteristics of a predetermined garment; a prediction state acquisition unit (56) that acquires information relating to a situation in which a predetermined exercise is performed as a prediction condition; a prediction unit (62) which, based on the physical information, the information about the garment and the prediction condition, predicts a user state that will result when the user performs the predetermined exercise by wearing the predetermined garment; and an output unit (46) that outputs information about the user state; wherein the virtual fitting device (100) also comprises an avatar generation unit (70) which generates an avatar image in which the user state is represented in an avatar modeled on the basis of the physical information, wherein the output unit (46) outputs the avatar image; and wherein the user information acquisition unit (52) also acquires exercise data to be obtained when the user has actually performed the predetermined exercise; and the virtual fitting device (100) also includes a reference motion capture unit (60) that captures reference motion information defining a standard movement of a person performing the predetermined exercise; the prediction unit (62) predicts a user movement that will be performed when the user performs the predetermined exercise by wearing the predetermined garment, based on information about the garment, exercise data and reference movement information; and the avatar generation unit (70) generates the avatar image to represent a situation in which the avatar performs the predetermined exercise with the user's movement; wherein the prediction unit (62) as the user state predicts a user state after the exercise that results after the user has continued the predetermined exercise for a predetermined period of time by wearing the predetermined garment.

Description

BACKGROUND Technical Area

[0001] The present invention relates to a virtual fitting device. Background information

[0002] A virtual fitting technique is known in which a composite image of a user wearing a garment is generated, allowing the user to virtually experience the fitting even without actually wearing the garment. For example, JP 2006-249618 A describes a technique in which information about the movement of a person being dressed is used for a fitting trial to estimate the movement of the garment that would be performed when the person is wearing it, and an image of the garment performing the estimated movement is combined with a moving image of the person being dressed. Thus, the technique described in JP 2006-249618 A presents the person being dressed with a video image showing movements that closely approximate a state in which the person was wearing the actual garment.US 2016 / 0284017 A1 relates to a system for generating photorealistic images of a virtual garment, which is superimposed on a visual image linked to a specific photographic subject using three-dimensional virtual components. US 10665022 B2 relates to an augmented reality display system for overlaying clothing and fitness information. US 9704296 B2 relates to an image processing system, specifically for creating a morphable 3D replica of a fully clothed person. CN 113467259 A relates to a smart device and a body management method for measuring and managing the user's body parameter information. SUMMARY

[0003] However, the technique described in JP 2006-249618A does not take into account the situation in which an exercise is performed when the person to be dressed is wearing the garment. Therefore, there is room for improvement in the evaluation of the garment.

[0004] The present invention was conceived in view of such circumstances, and one objective of the present invention is to provide a virtual fitting technique with which it is possible to accurately evaluate a garment.

[0005] A virtual fitting device according to one aspect of the present invention is defined in claim 1. A virtual fitting device according to another aspect of the present invention is defined in claim 6. A virtual fitting device according to yet another aspect of the present invention is defined in claim 8.

[0006] Another aspect of the present disclosure is a virtual fitting program. The program causes a computer to implement the following: a function for acquiring physical information about a user; a function for acquiring garment information relating to the properties of a predetermined garment; a function for acquiring information relating to a situation in which a predetermined exercise is performed, as a prediction condition; a function for predicting a user state that will result when the user performs the predetermined exercise by wearing the predetermined garment, based on the physical information, the garment information, and the prediction condition; and a function for outputting information relating to the user state.

[0007] It should be noted that any combinations of the above-mentioned components and those obtained by mutual substitution of the components and expressions of the present invention under a method, a device, a program, a transitory or non-transitory storage medium storing a program, and a system shall also be considered aspects of the present invention.

[0008] According to one aspect of the present invention, it is possible to provide a virtual fitting technique with which a garment can be accurately evaluated. BRIEF DESCRIPTION OF THE DRAWINGS Fig. Figure 1 is a schematic representation of a virtual fitting system. Fig. Figure 2 is a functional block diagram showing a schematic configuration of the virtual fitting system. Fig. Figure 3 is a function block diagram showing a schematic configuration of an arithmetic part of a virtual fitting server. Fig. Figure 4 is a functional block diagram showing a schematic configuration of a user information capture unit. Fig. Figure 5 is a function block diagram showing a schematic configuration of a prediction unit. Fig. Figure 6 shows an example of a menu screen for a virtual try-on mode. Fig. Figure 7 shows an example of a screen for entering user information. Fig. Figure 8 shows an example of a selection screen for a face image that is included in an avatar. Fig. Figure 9 shows an example of a selection screen for choosing a target garment. Fig. Figure 10 shows an example of a screen displaying an avatar image. Fig. Figure 11 shows an example of a screen displaying an avatar image. DETAILED DESCRIPTION

[0009] The following descriptions illustrate embodiments with reference to the drawings. In the embodiments and variations, identical or equivalent components are designated with the same reference numerals, and any duplicate description is omitted.

[0010] Fig. Figure 1 is a schematic diagram showing the outline of a virtual fitting system 100. The virtual fitting system 100 mainly comprises a user terminal 10 and a virtual fitting server 20. The virtual fitting system 100 may also include a garment information server 2, or the function of the garment information server 2 may be integrated into the virtual fitting server 20. Similarly, the virtual fitting system 100 may also include a reference movement information server 4, or the function of the reference movement information server 4 may be integrated into the virtual fitting server 20. The individual elements of the virtual fitting system 100 are interconnected via wireless communication 16 and a network 18.

[0011] Both the user terminal 10 and the virtual fitting server 20 can be configured by a mobile device or a computer consisting of a central processing unit (CPU), a graphics processing unit (GPU), random access memory (RAM), read-only memory (ROM), an additional storage device, a display device, a communication device, and the like, as well as a program stored on the mobile device or computer. The configuration can be implemented, for example, such that a program executed by the virtual fitting server 20 is used by the user terminal 10 via the network 18.Alternatively, the configuration can be one in which the function of a virtual fitting system is implemented by a single device that has the functions of both the user terminal 10 and the virtual fitting server 20, and the user can execute a virtual fitting program by directly operating the device. The single device can be a personal computer, a mobile device such as a smartphone, or an information terminal such as a tablet. Furthermore, the single device can be implemented as a terminal installed in a store that sells clothing and is operated by an assistant, i.e., a salesperson, using a program stored on the terminal.

[0012] In this description, a “virtual fitting device” can refer to the virtual fitting system 100 as a whole or to the virtual fitting server 20. In the present embodiment, the virtual fitting server 20 essentially corresponds to the “virtual fitting device” because the configuration is such that many of the characteristic functions included in the “virtual fitting device” are provided in the virtual fitting server 20. However, the characteristic functions of the “virtual fitting device” can be decentralized between the user terminal 10 and the virtual fitting server 20, or the configuration can be implemented such that many of the functions are assigned to the user terminal 10.

[0013] With reference to Fig. Section 1 describes an example of the operation of the virtual fitting system 100. The user uses the user terminal 10 to input physical information about themselves, label information of a garment the user wishes to virtually try on (hereinafter referred to as the "target garment"), and information relating to a situation in which an exercise is performed (hereinafter referred to as the "target exercise"), which is assumed to be performed by wearing the target garment. The label information of the target garment includes, for example, the product number information of the target garment. The information relating to the situation in which the target exercise is performed is used as a prediction condition for predicting a user state, which is described later.

[0014] The user terminal 10 transmits the user's physical information, the target garment's labeling information, and the predicted state to the virtual fitting server 20 via wireless communication 16 and the network 18. The virtual fitting server 20 acquires information about the target garment's properties (hereinafter referred to as "garment information") from the garment information server 2. The virtual fitting server 20 predicts the user's state that will result when the user performs the target exercise by wearing the target garment, based on the acquired physical information, the garment information, and the predicted conditions. For example, in a case where the target exercise is running, the user's state includes information such as the target garment's breathability, body stress, and similar factors.

[0015] The virtual fitting server 20 models an avatar based on the user's physical information. The virtual fitting server 20 generates an avatar image that reflects the predicted user state. The virtual fitting server 20 transmits the generated avatar image to the user terminal 10. The user terminal 10 displays the received avatar image. As described above, the virtual fitting system 100 can display the user state that results when the user performs a predetermined exercise by wearing a predetermined fitting item, by reflecting the user state in the avatar modeled based on the user's physical information.

[0016] Fig. Figure 2 is a functional block diagram showing a schematic configuration of the virtual fitting system 100. Each functional block, which is shown in each of the figures including Fig. The function shown in Figure 2 can be implemented as hardware by an element or mechanical device such as a computer's processor or memory, or as software by a computer program or similar. Functional blocks are shown here that are implemented through the interaction of these elements. These functional blocks can be implemented in various forms using hardware, software, or a combination of both.

[0017] The user terminal 10 is equipped with an operating unit 30, a display unit 32, a communication unit 34, and a storage unit 36. The operating unit 30 receives a user operation. It receives the selection or input of, for example, physical information about the user, the labeling information of the target garment, the prediction condition, and the like. The display unit 32 displays information received from the virtual fitting server 20, i.e., information such as an avatar image, rating information, and recommendation information, which will be described later. The operating unit 30 receives, for example, an operating instruction in response to the information displayed on the display unit 32. The operating unit 30 and the display unit 32 can be integrally structured, for example, by means of a touch panel as hardware.For example, storage unit 36 ​​stores physical information about the user, exercise data to be obtained when the user has actually performed a predetermined exercise, including the target exercise, and information relating to the user's preferences. Details of this information are described later.

[0018] The communication unit 34 transmits the information selected or entered by the operating processing unit 30 and the information stored in the storage unit 36 ​​via the network 18 to the virtual fitting server 20. The communication unit 34 receives information from the virtual fitting server 20 and sends the information to the display unit 32. The communication unit 34 can be configured as hardware with a wireless communication module for wireless LAN communication, mobile phone communication, or the like.

[0019] The virtual fitting server 20 is equipped with a communication unit 40, a processing unit 42, a storage unit 44, and an output unit 46. The communication unit 40 receives information transmitted by the user terminal 10 and sends the received information to the processing unit 42. The communication unit 40 also transmits information, such as a calculation result provided by the processing unit 42, to the user terminal 10. The communication unit 40 can be configured as hardware via a communication module of a wired LAN, etc.

[0020] The arithmetic unit 42 performs arithmetic processing based on information received from the communication unit 40. Details of the arithmetic processing performed by the arithmetic unit 42 are described later. The arithmetic unit 42 stores the result of the calculation in the memory unit 44. The result of the calculation stored in the memory unit 44 is output by the output unit 46 and transmitted to the user terminal 10 via the communication unit 40.

[0021] Garment Information Server 2 stores garment information about one or more types of garments, including the target garment, along with identification information such as the ID. The Computing Unit 42 of the Virtual Fitting Server 20, via the Communication Unit 40, requests Garment Information Server 2 to transmit the garment information of the target garment based on the target garment label information received from the User Terminal 10. Garment Information Server 2 transmits the garment information of the target garment to the Virtual Fitting Server 20 in response to the request from the Virtual Fitting Server 20. Details of the garment information are described later.

[0022] The reference movement information server 4 stores information (hereinafter referred to as "reference movement information") that defines a standard movement of a person performing one or more types of exercises, including the target exercise. The computing unit 42 of the virtual fitting server 20, via the communication unit 40, requests the reference movement information server 4 to transmit the reference movement information of the target exercise based on information relating to a situation in which the target exercise is performed, i.e., the prediction condition received from the user terminal 10. The reference movement information server 4 transmits the reference movement information of the target exercise to the virtual fitting server 20 in response to the request from the virtual fitting server 20. Details of the reference movement information are described later.

[0023] Fig. Figure 3 is a functional block diagram showing a schematic configuration of the computing unit 42 of the virtual fitting server 20. The computing unit 42 is equipped with an information acquisition unit 50, a prediction unit 62, an avatar generation unit 70, an evaluation unit 64, and a recommendation unit 66.

[0024] The information acquisition unit 50 is equipped with a user information acquisition unit 52, a garment information acquisition unit 54, a predictive state acquisition unit 56, a time span acquisition unit 58, and a reference motion acquisition unit 60. The user information acquisition unit 52 acquires information relating to the user of the user terminal 10.

[0025] Fig. Figure 4 is a functional block diagram showing a schematic configuration of the user information acquisition unit 52. The user information acquisition unit 52 of the present embodiment is equipped with a physical information acquisition unit 82, a training data acquisition unit 84, a preference information acquisition unit 86, and a feedback acquisition unit 88.

[0026] The Physical Information Capture Unit 82 captures physical information about the user. This includes, for example, height, weight, body fat percentage, fasting or postprandial blood glucose levels, body temperature, and the size and shape of the body part on which the target garment is worn. For example, if the target garment is shoes or socks, the user's physical information may also include information about foot shape, such as foot length and circumference. If the target garment is clothing, the user's physical information may also include information about body type. The user's physical information may also include information about the injury history of the body part on which the target garment is worn.The physical information acquisition unit 82 can read image information, e.g., a photograph of the user's entire body or a specific body part, and determine the physical information about the user based on the image information.

[0027] The Exercise Data Acquisition Unit 84 acquires exercise data. The exercise data to be acquired by the Exercise Data Acquisition Unit 84 can be data recorded when the user actually performed the target exercise, or data estimated from data recorded when the user performed an exercise other than the target exercise. Alternatively, the exercise data to be acquired can be estimated from the user's physical information. The target exercise can be any sport. In this description, sports include not only competitive sports with relatively high impact, such as running, ball games, swimming, cycling, skiing, snowboarding, and skateboarding, but also exercises with relatively low impact, such as walking and stretching.

[0028] The exercise data can contain various types of measurement data that are collected when the user actually performs the target exercise. For example, the exercise data can include data on movement, measured by a motion sensor such as a nine-axis sensor. The exercise data can also include biological data, measured by a biological sensor such as a heart rate sensor or a body temperature sensor. The exercise data can include data such as position coordinates and altitude, measured by position information sensors such as a position sensor and an altitude sensor. The exercise data can include data such as temperature, air pressure, humidity, and wind speed, measured by environmental sensors such as a temperature sensor and a barometric pressure sensor. The exercise data can also include weather information, retrieved via the internet or similar sources.The exercise data is not limited to data obtained from a single measurement, but can also be data measured multiple times as the user performs a sequence of the target exercise. Alternatively, the exercise data can be data measured at the time the user performs each target exercise when performing the target exercise multiple times.

[0029] If the target exercise is running or walking, the exercise data may include, for example, stride length, step frequency (which refers to the number of steps per unit of time and is also called cadence), the intensity of a landing impact, a running or walking distance, time required, heart rate, maximum oxygen uptake, an analytical value in relation to form, a running or walking distance, altitude gained, and similar information.

[0030] The exercise data acquisition unit 84 can obtain exercise data from a server that manages already available applications or internet services for recording the exercise data.

[0031] The Preference Information Capture Unit 86 captures information related to a user's preference. User preference information refers to the user's preferences regarding the target garment. This information might include, for example, whether the user prefers a larger or smaller size, whether the user prefers a harder or softer feel (if the target garment is footwear), a preferred color, and a preferred design. User preference information can also include clothing history, linking information about a garment worn by the user in the past with information about the user's impression of the garment.The user's impression can include information about a snug fit, information about an area prone to wear and tear, and similar details. Information about the user's preferences can include the desired performance of the garment, i.e., whether the user prioritizes functionality or, for example, fashion. This user preference information can be entered by the user via user terminal 10 or generated based on past purchase history, search history, or similar data related to the user's clothing.Alternatively, information about user preferences could be obtained through a sensitivity survey, questionnaire, or similar method. This user preference information could be estimated based on information related to an action, such as a social media post or activity, or information related to the content of an account to be followed, and so on.

[0032] The feedback acquisition unit 88 captures feedback information entered by the user after they have performed the target exercise by wearing a different garment than the target garment. This feedback information is used by the prediction unit 62 to predict the user's condition and can be used to improve prediction accuracy.

[0033] Returning to Fig. 3: The Garment Information Capture Unit 54 gathers information about the garment that relates to the properties of the target garment. Target garments include, for example, athletic wear. Target garments include shoes, shoe inserts, socks, bandages, compression garments, clothing, suspenders, braces, and similar items. The garment information can include various types of information, such as the shape of the target garment, the quality of the material used for each part, the weight, the pressure exerted on the garment, the color and size, and other information that may affect performance in the target exercise. The Garment Information Capture Unit 54 can process the garment information that corresponds to the labeling information (e.g.,the product number) of the target garment, which are to be received by the user terminal 10, are captured by the garment information server 2.

[0034] The predictive state acquisition unit 56 captures information as a prediction condition relating to the situation in which the target exercise is performed. This prediction condition is used when the predictive unit 62 predicts the user state. The prediction condition can include information about the environment in which the target exercise is performed, details of the exercise, and similar information. For example, in a case where the target exercise is running, the prediction condition might include information about a route, such as whether the route is paved or unpaved, information about the climate, wind speed, running track, pace, and so on. In a case where the target exercise is soccer, the prediction condition might include information about the surface type of a soccer field, such as...B. Information about whether the football pitch consists of natural grass, artificial turf or earth.

[0035] The time span detection unit 58 collects information about a time span. Although details are described later, the time span information collected by the time span detection unit 58 is used by the prediction unit 62 to generate post-wear information. That is, the time span information collected by the time span detection unit 58 is used as information about a predetermined time span, which is to be used when the prediction unit 62 predicts a user post-exercise state that will result after the user has continued the target exercise for a predetermined time span by wearing the target garment.

[0036] The Reference Motion Acquisition Unit 60 acquires reference motion information. This reference motion information is information about a motion model defined for each type of target exercise. For example, if the target exercise is running, the reference motion information might be information relating to a human body model running in a standard form along a time axis. Although details are described later, the reference motion information is used as the reference motion when the Prediction Unit 62 predicts a user motion. The Reference Motion Acquisition Unit 60 can acquire the reference motion information, corresponding to the target exercise determination information received from the User Terminal 10, from the Reference Motion Information Server 4.

[0037] Based on the physical information acquired by the physical information acquisition unit 82, the garment information acquired by the garment information acquisition unit 54, and the prediction condition acquired by the prediction state acquisition unit 56, the prediction unit 62 predicts the user state that will result when the user performs the target exercise by wearing the target garment.

[0038] The user state predicted by the Prediction Unit 62 can contain information about various effects the user achieves when performing the target exercise while wearing the target garment. For example, the user state can include information about the physical properties of the target garment and information about the user's body, such as the garment's breathability, the pressure exerted on a garment component, the temperature distribution, the body's load, the stress generated at the joints, the degree of muscle fatigue, and the amount of perspiration. The user state is information that can be used to evaluate the target garment.

[0039] Fig. Figure 5 is a functional block diagram showing a schematic configuration of the prediction unit 62. The prediction unit 62 of the present embodiment is equipped with a user movement prediction unit 90, a posture change prediction unit 92, a movement change prediction unit 94, and a clothing change prediction unit 96.

[0040] The User Movement Prediction Unit 90 predicts a user movement that will be performed when the user performs the target exercise by wearing the target garment, based on the garment information of the target garment captured by the garment information capture unit 54, the exercise data captured by the exercise data capture unit 84, and the reference movement information captured by the reference movement capture unit 60. The user movement predicted by the User Movement Prediction Unit 90 is an example of the user state predicted by the Prediction Unit 62.

[0041] Specifically, the User Motion Prediction Unit 90 predicts the movement the user will perform while conducting the target exercise based on a difference between the reference motion data and the exercise data. The exercise data can be used as a parameter for predicting a user-specific movement by reflecting a user habit or characteristic in a standard movement based on the reference motion information. Next, the User Motion Prediction Unit 90 makes an adjustment based on the garment information of the target garment so that the predicted user movement corresponds to a movement in a state in which the target garment is worn.For example, if the target garment is shoes and the weight of the shoes is greater than that of normal shoes, the incline of the user's movement can be reduced or the shape modified. The extent to which the user's movement is modified according to the garment's information can vary depending on the exercise data. For example, the user's movement may be more dependent on the weight of the target garment if the user's muscle strength is weaker.

[0042] Posture Change Prediction Unit 92 predicts a change in the user's posture caused by the user wearing the target garment and continuing the target exercise for a predetermined period. Movement Change Prediction Unit 94 predicts a change in the user's movement caused by the user wearing the target garment and continuing the target exercise for a predetermined period. Clothing Change Prediction Unit 96 predicts a change in the target garment caused by the user wearing the target garment and continuing the target exercise for a predetermined period.

[0043] In this description, the predetermined time span can be a time span recorded by the time span recording unit 58 or a preset time span. The preset time span can be information about a period of time that is associated in advance with at least one of the two elements: the target garment and the target exercise. The predetermined period can vary depending on the target exercise and can range, for example, from a short period such as a sequence of exercises (a game, a race, or the like) to a long period such as several years. "Continuing the target exercise for a predetermined period" is not limited to continuing the target exercise without interruption but includes the habitual performance of the target exercise. Furthermore, the predetermined period is not limited to a fixed time such as one hour but can be a period until a specific condition is met, e.g.,a period of time until a game is decided, a period of time until a running competition is finished, or something similar.

[0044] The Posture Change Prediction Unit 92 predicts a change in the user's posture based on reference motion information captured by the Reference Motion Capture Unit 60, exercise data captured by the Exercise Data Capture Unit 84, and garment information of the target garment captured by the Garment Information Capture Unit 54. For example, based on the reference motion information and the exercise data, the Posture Change Prediction Unit 92 predicts how much the user's posture will change if the user continues the target exercise by wearing the target garment for a predetermined period of time.For example, based on the reference movement information, exercise data, and garment information of the target garment, the posture change prediction unit 92 predicts the effect that a change in a support function of the target garment will have on a change in the user's posture to be maintained after the user has continued the target exercise for a predetermined period of time by wearing the target garment.

[0045] A variety of parameters related to exercise data and a variety of parameters related to the garment information of the target garment can change individually or interactively depending on the duration of the target exercise. For example, if the target exercise is running or walking, the exercise data includes a stride length parameter and a stomp parameter. The stride length and stomp parameters tend to decrease as the duration of the target exercise increases. The user's exercise data also includes information about individual variation in the degree of change over time for each parameter. That is, the extent of change over time for the stride length parameter and the extent of change over time for the stomp parameter will differ from user to user.The garment information includes a weight parameter. This weight parameter remains constant regardless of the duration of the target exercise. However, the larger the weight parameter, the more likely the user is to fatigue. Therefore, the stride length and stomp parameters, which are part of the exercise data, may decrease. As described above, the Posture Change Prediction Unit 92 can predict a change in the user's posture by using the information about a change in each parameter over time. The same applies to the prediction of a change in user movement, performed by the Movement Change Prediction Unit 94, and to the prediction of a change in the target garment, performed by the Garment Change Prediction Unit 96.

[0046] In a case where the target exercise is, for example, running, a change in the user's posture is a change in running form. The Posture Change Prediction Unit 92, for instance, predicts how much the hips will drop and the upper body will lean forward, or similar changes, in the final stages of a long-distance run, such as a marathon. The Posture Change Prediction Unit 92 can also predict a change in the shape of the part of the body where the target garment is worn, such as shoes, swollen feet, or similar issues, as a change in the user's posture.The Posture Change Prediction Unit 92 can predict, based on reference movement information, exercise data, and garment information about the target garment, a change in the user's facial expression or the like caused by the user wearing the target garment and continuing the target exercise for a predetermined period of time.

[0047] The motion change prediction unit 94 predicts a change in user motion based on reference motion information captured by the reference motion capture unit 60, exercise data captured by the exercise data capture unit 84, and garment information captured by the garment information capture unit 54 about the target garment. For example, based on the reference motion information and the exercise data, the motion change prediction unit 94 predicts a change in motion due to user fatigue caused by the user continuing the target exercise by wearing the target garment for a relatively short period of time.For example, based on reference movement information, exercise data, and garment information about the target garment, the movement change prediction unit 94 predicts an influence that the user's habituation or familiarity with the target garment will have on the user's movement to be maintained after the user has continued the target exercise by wearing the target garment for a relatively long period of time.

[0048] The garment change prediction unit 96 predicts a change in the target garment described above based on the reference motion information captured by the reference motion capture unit 60, the exercise data captured by the exercise data capture unit 84, and the garment information of the target garment captured by the garment information capture unit 54. For example, based on the reference motion data, the exercise data, and the garment information of the target garment, the garment change prediction unit 96 predicts a change in the elasticity of the target garment and a change in properties, such as a supporting force on a part that comes into contact with the user's body, caused by the user continuing the target exercise by wearing the target garment for a predetermined period of time.In a case where it is assumed that the target garment will not change significantly over a short period of time, the garment change prediction unit 96 can predict that the target garment will not change if the predetermined period of time is a period shorter than or equal to a predetermined threshold period of time.

[0049] As described above, the posture change prediction unit 92, the movement change prediction unit 94, and the garment change prediction unit 96 predict changes in the user's posture, movement, and target garment caused by the user wearing the target garment and continuing the target exercise for a predetermined period. Based on a prediction outcome of changes in the user's posture, movement, and / or target garment caused by the user wearing the target garment and continuing the target exercise for a predetermined period, the prediction unit 62 predicts the user's post-exercise state after the user has continued the target exercise for a predetermined period while wearing the target garment.The user state after exercise is an example of user state. The user state after exercise can be identical to the information in the prediction result above. Prediction Unit 62 can predict, as the user state after exercise, any of the incremental changes in user state that occur when the user continues the target exercise for a predetermined period. For example, in a case where the target exercise is running, if the predetermined period is a marathon in a single race, Prediction Unit 62 can predict each of the changes in user state for each 5 km run by the user as the user state after exercise.

[0050] The user's post-exercise condition can contain information about a performance indicator of the target exercise. For example, if the target exercise is running, the performance indicator information could be an analytical value relating to running form, a predicted race time, and so on. The performance indicator information is not limited to an absolute value but can also be a relative value indicating how much the performance indicator changes compared to other clothing items.

[0051] The user's condition after the exercise can provide an indicator of the fit, indicating how well the target garment is adapted to the user.

[0052] Predictor Unit 62 can continue to predict the user's state based on feedback information collected by Feedback Capture Unit 88. As described above, this feedback information refers to feedback provided by the user after performing the target exercise by wearing a different garment than the target garment. Since Predictor Unit 62 predicts the user's state based on this feedback information, as described above, the feedback information complements the physical information about the user, thus improving prediction accuracy.

[0053] The User Movement Prediction Unit 90 can correct the predicted user movement based on a prediction result of the change in the user's posture predicted by the Posture Change Prediction Unit 92, the change in user movement predicted by the Movement Change Prediction Unit 94, and the change in the target garment predicted by the Garment Change Prediction Unit 96. Accordingly, the User Movement Prediction Unit 90 can predict the user movement that will result after the user has continued the target exercise for a predetermined period of time while wearing the target garment.

[0054] The prediction unit 62 can use a predictive model learned through machine learning. In this case, the predictive model outputs the user state, including user movement and the user state after exercise, when physical information, clothing information, a prediction condition, exercise data, reference movement information, time span information, and feedback information are input. The prediction unit 62 can also use an analytical method such as regression analysis or multivariate analysis for its prediction.In this case, the physical information, the garment information, the prediction condition, the exercise data, the reference movement information, the time span information, and the feedback information can be assumed to be explanatory variables, and the user state, including the user movement and the user state after the exercise, can be assumed to be target variables.

[0055] Returning to Fig. 3: The avatar generation unit 70 is equipped with a modeling unit 72 and an image generation unit 74. The modeling unit 72 models an avatar based on the physical information about the user. The avatar is a figure represented in a virtual space as the user's alter ego and can be modeled using a known technique. The avatar can represent the user's entire body, a specific body part (e.g., the part on which the target clothing item is worn), or both. The avatar can realistically represent the user or depict them by emphasizing one of their features. Furthermore, the user can be represented in a simplified form, such as a stick figure or similar, or by a figure, such as an animal. The modeling unit 72 can model a part of the avatar, e.g.,Model a facial type based on the information selected by the user.

[0056] Modeling Unit 72 reflects the physical information about the user in the avatar. For example, if the avatar represents the user's entire body, Modeling Unit 72 models an avatar of the body type that corresponds to the user's height, weight, body fat percentage, etc. If, for example, the avatar represents a specific body part of the user, an avatar is modeled that corresponds to the size of that body part.

[0057] Modeling unit 72 can also model an avatar based on garment information about the target garment, in addition to physical information about the user. In this case, modeling unit 72 models an avatar in a state where the target garment is being worn. The avatar can change depending on the type, size, shape, color, etc., of the target garment.

[0058] The image generation unit 74 generates an avatar image that reflects the user state predicted by the prediction unit 62 in the avatar modeled by the modeling unit 72. For example, if the user state indicates information about the physical properties of the target garment, such as breathability, pressure, temperature distribution, or similar, the image generation unit 74 generates an avatar image that changes a display mode, such as a color distribution for the body part on which the target garment is worn. Similarly, if the user state indicates information about the user's body, such as muscle fatigue level, amount of perspiration, or similar, the image generation unit 74 generates an avatar image that changes a display mode, such as a color distribution for the entire body or body part, or a magnified view of a target location.

[0059] If the user state predicted by the prediction unit 62 is the post-exercise state, the image generation unit 74 generates an avatar image representing the post-exercise state. The avatar image reflecting the user's post-exercise state can be displayed in a similar mode to the avatar image reflecting the user state described above. If the post-exercise state predicted by the prediction unit 62 involves a variety of incremental changes in the user state, the image generation unit 74 can generate an avatar image corresponding to each of the changes in the user state.

[0060] Once the user motion prediction unit 90 has predicted the user motion, the image generation unit 74 generates an avatar image to depict a situation in which the avatar performs the target exercise with the user motion. Accordingly, the image generation unit 74 can reproduce information in the avatar image about how the user moves when performing the target exercise by wearing the target garment.

[0061] The avatar image can consist of still image data, moving image data, or 3D modeling data. In the case where the avatar image is 3D modeling data, it can be converted into a data format that can be displayed when the avatar image is shown on the user terminal 10 or similar device after being output by the output unit 46. The image generation unit 74 can use either a third-person view image of the avatar or a view image of the avatar itself as the avatar image. For example, if the avatar image is a view image of the body part on which the target garment is worn, it will show the view. Such an image allows the user to easily visualize the state they will be in when wearing the garment.

[0062] The evaluation unit 64 generates evaluation information about the target garment based on the user state predicted by the prediction unit 62. Evaluation unit 64 can evaluate information contained in the user state information, such as any information relating to the physical properties of the target garment and information relating to the user's body, and generate evaluation information that includes an evaluation of each item and a comprehensive evaluation. The evaluation information can include, for example, information about a score or rank according to the comprehensive evaluation and information about a point distribution according to the evaluation of each item.The evaluation information can be displayed as a diagram or a numerical value, or the image generation unit 74 described above can reproduce the evaluation information in the avatar image. This means, for example, that the avatar's expression can be changed according to the comprehensive evaluation.

[0063] Recommendation Unit 66 generates recommendation information that displays a garment recommended to the user to wear based on the garment information for each of the multitude of garments. In this case, the virtual fitting server 20 captures the garment information for each of the multitude of garments, for example, through the following processing. The garment information capture unit 54 captures the garment information for each of the multitude of garments from the garment information server 2. The garment information to be captured by the garment information capture unit 54 can be garment information that is designated as the target garment, or garment information that meets a predetermined condition if the target garment is not specified.Prediction Unit 62 predicts the user's state for each of the multiple garments. Evaluation Unit 64 generates evaluation information for each of the multiple garments based on the user's state with respect to each garment. Based on the evaluation information obtained in this way for each of the multiple garments, Recommendation Unit 66 can, for example, generate information about a garment with the highest overall rating with respect to the evaluation information as recommendation information to advise the user to wear it. It should be noted that Image Generation Unit 74 can generate each of the avatar images that reflect the user's state with respect to each of the multiple garments predicted by Prediction Unit 62.In this way, the avatar images for the multitude of clothing items can be displayed simultaneously, for example to facilitate the comparison of the clothing items.

[0064] Returning to Fig. 2: Storage unit 44 stores the calculation results, such as the avatar images generated by processing unit 42. Storage unit 44 can pre-store information that processing unit 42 uses for its calculations.

[0065] Output unit 46 displays the avatar image generated by avatar generation unit 70. Output unit 46 can output information relating to the user state predicted by prediction unit 62 in ways other than the avatar image, such as information represented by characters, numerical values, or graphs. However, outputting the avatar image is preferable because the user can intuitively understand the user state. Output unit 46 can display the evaluation information generated by evaluation unit 64. Output unit 46 can display the recommendation information generated by recommendation unit 66. The information output by output unit 46 is transmitted to user terminal 10 via communication unit 40.

[0066] Fig. 6, Fig. 7, Fig. 8, Fig. 9, Fig. 10 to Fig. 11 are examples of display screens shown by the display unit 32 of the user terminal 10. Fig. Figure 6 shows an example of a menu screen for a virtual try-on mode. If a Fig. When the "Create Avatar" option is selected (see image 6), the display unit 32 switches to a view in Fig. 7 displayed screen.

[0067] Fig. Figure 7 shows an example of a screen for entering user information. If this is in Fig. When the option shown in Figure 124, “Input of physical information,” is selected, the user terminal 10 receives input of physical information, such as body type and size. If the Fig. When the “Measurement Data” option shown in Figure 122 is selected, the virtual fitting server 20 captures the user's exercise data, e.g., from a server managing third-party services. If this is the case, the following applies: Fig. When the “Generate” option shown in Figure 126 is selected, the user terminal 10 transmits the entered physical information to the virtual fitting server 20, and the virtual fitting server 20 generates an avatar based on the physical information and the exercise data.

[0068] Fig. Figure 8 shows an example of a selection screen for a face image in an avatar. As in Fig. As shown in section 8, the user can change the facial image contained in the avatar.

[0069] If one in Fig. When image 114 “Shoe selection” is selected, the display unit 32 switches to a view shown in 6. Fig. 9 displayed screen. Fig. Figure 9 shows an example of a selection screen for choosing a target garment. Fig. 9 displays one or more shoes as the target garment for selection. The user terminal 10 transmits information identifying the selected shoes as the target garment to the virtual fitting server 20.

[0070] Images corresponding to a variety of exercise types are displayed below a heading. Fig. The “virtual try-on” image 116 shown in Figure 6 is displayed. For example, if a “Running” image 118 is selected, the user terminal 10 transmits information identifying the target exercise as running to the virtual try-on server 20. The user terminal 10 also receives input of information relating to a situation in which running is performed as the target exercise and transmits the input information as a prediction condition to the virtual try-on server 20.

[0071] The virtual fitting server 20, in the prediction unit 62, predicts the user state that will result when the user wears the shoes designated as the target garment and performs the running activity designated as the target exercise, based on the input physical information, the garment information, and the prediction conditions. Additionally, in the avatar generation unit 70, the virtual fitting server 20 generates an avatar image by reflecting the predicted user state in the avatar modeled based on the physical information.

[0072] Fig. 10 and Fig. Figures 11 each show an example of a screen displaying the generated avatar image. The figures demonstrate that when the user performs the target exercise under the prediction condition, the target clothing item, which represents the avatar image of Fig. 10 represents, is better than the target garment, which is the avatar image of Fig. 11 represents the fact that the user is less likely to tire, the shape can be more stable, and the garment feels comfortable to wear.

[0073] Here, a specific example of the use of the virtual fitting system 100 of the present embodiment is described. First, a case is described in which the target sport is soccer. In this case, the physical information about the user includes information about their height, weight, and foot shape. The target apparel is shoes, and the apparel information includes information about the materials and structures of an upper and outsole. The prediction condition includes information about the surface type of a soccer field, e.g., information about whether the soccer field is natural grass, artificial turf, or dirt. In the present example, no practice data may be used.

[0074] The prediction unit 62 of this example, based on user state, predicts information about the pressure exerted on the feet and shoes at the time of a step or foot movement. The avatar generation unit 70 of this example models the entire body and / or feet as an avatar and generates an avatar image in which the pressure information predicted by prediction unit 62 is displayed through a color change or similar means. This makes it possible to predict the level of pressure exerted on the feet by the target garment.

[0075] Next, a case is described where the target exercise is running. In this case, the user's physical information includes details about their height, weight, and foot shape. The exercise data includes information about the environment in which the exercise is performed and the movement itself. The target garment is either shoes or running apparel, and the garment information includes details about size, material, and structure if the target garment is shoes. The material includes information about the upper material and the sole material. The structure includes information about the last, midsole, laces, and upper. In the case of athletic wear, the garment information includes details about the size, cut, and fabric material.The forecast condition includes information about the type of route, temperature, humidity, season, wind speed, running distance, running speed, running style, and hunger level.

[0076] Prediction Unit 62 of this example predicts user state information such as muscle fatigue, body strain, and the breathability of the running apparel. If, for example, both shoes and running apparel are selected as target garments, Prediction Unit 62 also incorporates the interaction between the two into the user state. Avatar Generation Unit 70 of this example models the entire body and / or feet as an avatar and generates an avatar image in which the information predicted by Prediction Unit 62 is displayed through color changes, changes in the size of the body part, or similar actions. Consequently, it is possible to predict the perceived comfort of wearing the target garment over a medium to long period.

[0077] As described above, according to the present embodiment, the prediction unit 62 predicts the user state that will result when the user performs the target exercise by wearing the target garment, based on the physical information, the garment information, and the prediction condition. The avatar generation unit 70 generates an avatar image in which the user state is represented in the avatar modeled on the basis of the physical information. Consequently, it is possible to generate an avatar image that reflects the user state predicted according to a situation in which a predetermined exercise is performed. Therefore, the user can intuitively and accurately evaluate the garment by viewing the avatar image.

[0078] Furthermore, according to the present embodiment, the prediction unit 62 predicts, based on the garment information, the exercise data, and the information about the reference movement, the user movement that will be performed when the user carries out the target exercise while wearing the target garment. The avatar generation unit 70 generates an avatar image to represent a situation in which the avatar performs the target exercise with the user movement. Because an avatar image is generated that represents the user's movement and reflects the properties of the garment, the user is able to understand how the garment influences the movement.

[0079] Furthermore, according to the present embodiment, the prediction unit 62 predicts a post-exercise user state that results after the user has continued the target exercise for a predetermined period by wearing the target garment. More precisely, the prediction unit 62 predicts at least one change in the user's posture, one change in user movement, and one change in the target garment caused by the user wearing the target garment and continuing the target exercise for a predetermined period, and predicts the post-exercise user state based on a prediction result. Consequently, since it is possible to predict the state that will be reached after using the garment for a specific period, the garment can be evaluated more accurately.

[0080] The embodiment described above can be a program that causes a computer to implement the function for carrying out the method described above, or a recording medium for storing the program. The recording medium for storing such a program can be a non-transient and tangible computer-readable recording medium (storage medium). More specifically, the recording medium can be a magnetic recording medium, such as non-volatile memory, magnetic tape, or magnetic disk, or an optical recording medium, such as an optical disc.

[0081] The embodiments have been described above. It should be readily understood by those skilled in the art that these embodiments are merely examples and that various modifications can be made to combinations of the components and the methods for processing the embodiments, and that such modifications are also within the scope of the present invention. Furthermore, when the embodiments described above are generalized, the following aspects are obtained. [Aspect 1]

[0082] A virtual fitting device that includes the following: a user information capture unit that captures physical information about a user; a garment information capture unit that captures garment information relating to the characteristics of a predetermined garment; a predictive state acquisition unit that captures information as a predictive condition relating to a situation in which a predetermined exercise is performed; a prediction unit that, based on physical information, garment information, and the prediction condition, predicts a user state that will result when the user performs the predetermined exercise by wearing the predetermined garment; and An output unit that provides information about the user's state.

[0083] According to this aspect, the user state resulting from the user performing the predetermined exercise while wearing the predetermined garment is predicted and output based on physical information about the user, information regarding the properties of the predetermined garment, and information regarding the situation in which the predetermined exercise is performed. This makes it possible to accurately evaluate the garment based on the output user state.

[0084] The virtual fitting device according to aspect 1 also includes an avatar generation unit that generates an avatar image in which the user state is represented in an avatar modeled on the basis of physical information, wherein The output unit displays the avatar image.

[0085] Since the avatar image, which reflects the user's state in the avatar based on the physical information about the user, is generated and output, the user's state is intuitively easy to understand based on the output avatar image.

[0086] According to the virtual fitting device as described in Aspect 1, the user information acquisition unit also captures exercise data that is to be obtained when the user has actually performed the predetermined exercise; The virtual fitting device also includes a reference motion capture unit that captures reference motion information defining a standard movement of a person performing the predetermined exercise; Based on the garment information, exercise data, and reference movement information, the prediction unit predicts a user movement that will be performed when the user performs the predetermined exercise by wearing the predetermined garment; and The avatar generation unit creates the avatar image to represent a situation in which the avatar performs the predetermined exercise with the user's movement.

[0087] According to the present aspect, it is possible to depict a situation in the avatar image in which the user performs the prescribed exercise by wearing the prescribed garment.

[0088] According to the virtual fitting device according to aspect 1, the prediction unit predicts as the user state a user state after the exercise that results after the user has continued the predetermined exercise for a predetermined period of time by wearing the predetermined garment.

[0089] Since it is possible to predict the condition that will occur after a certain period of use of the garment, the garment can be evaluated more accurately according to this aspect. [Aspect 2]

[0090] The virtual fitting device according to aspect 1, wherein The prediction unit predicts at least one change in the user's posture, one change in user movement, and one change in the predetermined garment caused by the user wearing the predetermined garment and continuing the predetermined exercise for the predetermined period of time, and predicts the user's post-exercise condition based on a prediction result.

[0091] According to the present aspect, it is possible to predict the condition that will occur after the garment has been used for a specific period of time, based on a prediction result derived from at least one of the changes in the user's posture, the user's movement, and the garment itself. Therefore, the garment can be evaluated more accurately. [Aspect 3]

[0092] The virtual fitting device according to aspect 1 or 2, wherein the user state after the exercise contains information about a performance indicator of the predetermined exercise.

[0093] Since it is possible to predict the information about the exercise's performance indicator that can be obtained after using the garment over a certain period of time, the garment can be evaluated more accurately according to this aspect. [Aspect 4]

[0094] The virtual fitting device according to one of aspects 1 to 3, wherein the user's condition after the exercise contains information about the fit of the predetermined garment.

[0095] Since it is possible to predict the information about the fit of the garment that can be obtained after a certain period of use, the garment can be evaluated more accurately according to this aspect. [Aspect 5]

[0096] The virtual fitting device according to one of aspects 1 to 4, which also includes a time-span recording unit that captures information over a time span, wherein The prediction unit predicts the user's state after the exercise by considering the time span recorded by the time span detection unit as the predetermined time span.

[0097] Since it is possible to predict the condition that will occur after the garment has been used for a certain period of time, the garment can be evaluated more accurately according to this aspect. [Aspect 6]

[0098] The virtual fitting device according to aspect 1 also includes an evaluation unit that generates evaluation information about the predetermined garment based on the user's state, wherein the output unit further outputs the evaluation information.

[0099] By generating and outputting evaluation information about the garment, the garment can be assessed more objectively with regard to this aspect. [Aspect 7]

[0100] The virtual fitting device according to aspect 6, wherein: the garment information capture unit captures each piece of garment information from a large number of garments; the prediction unit predicts the user's condition for each of the many garments; The evaluation unit generates the evaluation information for each of the multitude of garments based on the user's status regarding each of the multitude of garments; The virtual fitting device also includes a recommendation unit that generates recommendation information, displaying a garment recommended to the user to wear based on the evaluation information about each of the multitude of garments; and the output unit that further outputs the recommendation information.

[0101] By generating and outputting recommendation information about the garment, information about the garment suitable for the user can be provided, according to the present aspect.

[0102] According to the virtual fitting device described in Aspect 6, the user information acquisition unit also captures preference information regarding a user preference; and The evaluation unit generates further evaluation information based on the preference information.

[0103] According to the present aspect, the accuracy of the evaluation information can be improved, since the evaluation information about the garment is created and output based on information about the user's preferences. [Aspect 8]

[0104] The virtual fitting device according to aspect 1 wherein: The user information capture unit also captures feedback information entered by the user after they have performed the predetermined exercise by wearing a different garment than the predetermined one; and The prediction unit further predicts the user's state based on the feedback information.

[0105] According to the present aspect, the accuracy of the prediction can be further improved, as the user state is further predicted based on the feedback information. [Aspect 9]

[0106] A virtual fitting program that causes a computer to implement the following: a function for capturing physical information about a user; a function for capturing information about the garment relating to the characteristics of a predetermined garment; a function for capturing information as a prediction condition relating to a situation in which a predetermined exercise is performed; a function for predicting, based on physical information, information about the garment, and the prediction condition, a user state that results when the user performs the predetermined exercise by wearing the predetermined garment; and a function for outputting information regarding the user's state.

[0107] According to this aspect, the user state resulting from the user performing the predetermined exercise while wearing the predetermined garment is predicted and output based on physical information about the user, information regarding the properties of the predetermined garment, and information regarding the situation in which the predetermined exercise is performed. This makes it possible to accurately evaluate the garment based on the output user state.

Images