System and method for virtual fitting during live broadcast

Abstract

Methods and systems for enhancing streaming media data to include virtual fitting data are described herein. The method includes evaluating the streaming data to obtain a first three-dimensional (3D) model associated with a product. The method also includes obtaining a second 3D model associated with a user. The first 3D model is then tried on the second 3D model, and the tried-on model is posed in a manner estimated from a presenter in the streaming data. The posed model is then rendered and presented to a viewer along with the streaming data. Embodiments of the present disclosure are applicable to various virtual reality applications and computer-based fitting systems.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application is based on the U.S. provisional patent application with application number 62 / 987,474, application date March 10, 2020, and application name “System and method for virtual fitting during live broadcast”, and claims the priority of the U.S. provisional patent application. The entire contents of the U.S. provisional patent application are hereby incorporated into this application by introduction. Technical Field

[0003] The present invention generally relates to methods and systems related to virtual fitting applications. More specifically, embodiments of the present invention provide methods and systems for enhancing streaming media data to include virtual fitting data. Background Art

[0004] When considering whether to purchase a product online, users often have difficulty visualizing how the product will look or perform. This is especially true when it comes to apparel or other wearable products. To better understand the attributes of a product, users often view pictures and / or videos of a demonstrator evaluating and reviewing the product. These pictures and videos show the product on the demonstrator while the demonstrator discusses the various pros and cons of the product. However, because people have different body shapes, even seeing the product on a demonstrator may not help the user fully visualize how the product will look or perform on him / her.

[0005] Embodiments of the invention address these and other problems individually and collectively. Summary of the invention

[0006] The method includes evaluating the streaming data to obtain a first three-dimensional (3D) model associated with a product. The method also includes obtaining a second 3D model associated with a user. The first 3D model is then tried on the second 3D model and the tried-on model is posed in a manner estimated from a presenter in the streaming data. The posed model is then rendered and presented to a viewer along with the streaming data. Embodiments of the present invention are suitable for various applications in virtual reality and computer-based fitting systems.

[0007] One embodiment of the present invention is directed to a method comprising receiving an indication of media content that a user is viewing, identifying a product associated with the media content, obtaining a first 3D model representing the product, obtaining a second 3D model representing the user, determining a presentation pose based on the media content, applying the presentation pose to the second 3D model, generating a third 3D model by having the second 3D model try on the first 3D model, and presenting the third 3D model to the user in the presentation pose.

[0008] Another embodiment of the present invention relates to a system, comprising a processor and a memory including instructions, which, when executed by the processor, at least cause the system to receive an indication of media content that a user is viewing, identify a product associated with the media content, obtain a first 3D model representing the product, obtain a second 3D model representing the user, determine a demonstration pose based on the media content, apply the demonstration pose to the second 3D model, generate a third 3D model by having the second 3D model try on the first 3D model, and present the third 3D model to the user in the demonstration pose.

[0009] Yet another embodiment of the present disclosure relates to a non-transitory computer-readable medium storing specific computer-executable instructions, which, when executed by a processor, at least cause a computer system to receive an indication of media content that a user is viewing, identify a product associated with the media content, obtain a first 3D model representing the product, obtain a second 3D model representing the user, determine a demonstration pose based on the media content, apply the demonstration pose to the second 3D model, generate a third 3D model by having the second 3D model try on the first 3D model, and present the third 3D model to the user in the demonstration pose.

[0010] The present system has a number of advantages over conventional systems. For example, embodiments of the present invention relate to methods and systems for providing a user with a more accurate assessment of the appearance of a garment or other wearable product on him / her. In the system, streaming media data is enhanced using virtual fitting data of the user. To this end, a product model associated with the streaming media data is identified, and a user model associated with a viewer of the streaming media data is obtained. The product model is tried on the user model, and the user model is posed in a manner similar to the outline of a presenter in the streaming media data. The product model and the user model are rendered and presented with the streaming media data (e.g., enhanced in the streaming media data). BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 An illustrative example of a system that can augment streaming video with virtual fitting information in accordance with at least some embodiments is shown.

[0012] Figure 2 A system architecture for a system for augmenting streaming data with virtual fitting information, according to at least some embodiments, is shown.

[0013] Figure 3 A simplified flow chart of a method for presenting a data stream enhanced with virtual fitting data according to an embodiment of the present invention is shown.

[0014] Figure 4 Illustrative examples of techniques for obtaining a 3D model using sensor data in accordance with at least some embodiments are shown.

[0015] Figure 5 An example of a graphical user interface (GUI) for demonstrating achievable features according to an embodiment of the present invention is shown.

[0016] Figure 6 A flow chart of a process for presenting virtual fitting data to a user is shown, according to at least some embodiments.

[0017] Figure 7 An example of components of a computer system according to some embodiments is shown.

[0018] Figure 8 A block diagram of an apparatus for presenting virtual fitting data to a user is shown, according to at least some embodiments. DETAILED DESCRIPTION

[0019] The present invention generally relates to methods and systems related to virtual reality applications. More specifically, embodiments of the present invention provide methods and systems for determining the degree of fit between a user and a product. Embodiments of the present invention are applicable to various applications in virtual reality and computer-based fitting systems.

[0020] Figure 1 An illustrative example of a system that can augment streaming video with virtual fitting information in accordance with at least some embodiments is shown. Figure 1 In the example, the user device 102 is used to provide a request for virtual fitting information to the mobile application server 104. In some cases, the user device can be used to obtain user data 106, which can be provided to the mobile application server 104 for generating the virtual fitting information.

[0021] In one example, the user device 102 represents a suitable computing device that includes one or more graphics processing units (GPUs), one or more general purpose processors (GPPs), and one or more memories storing computer-readable instructions that can be executed by at least one of the processors to implement various functions of embodiments of the present invention. For example, the user device 102 can be any one of a smartphone, a tablet computer, a laptop, a personal computer, a game console, or a smart TV. The user device 102 may also include a rangefinder camera (i.e., a depth sensor) and / or an RGB optical sensor (e.g., a camera).

[0022] The user device can be used to capture and / or generate user data 106. The user data 106 may include information related to a specific user (e.g., the user of the user device 102) for whom virtual fitting data should be created. The user data 106 may include data about the user that can be used to generate virtual fitting data. For example, the user data 106 may include the user's dimensions. The user data 106 may be captured in any suitable format. For example, the user data 106 may include a point cloud, a 3D mesh or model, or a string containing measurements at predetermined locations. In some cases, capturing the user data 106 includes receiving information about the user manually entered into the user device 102. For example, the user may enter measurements of various parts of the user's body via a keyboard. In some cases, acquiring the user data 106 may include using a camera and / or a depth sensor to acquire image / depth information related to the user. The user device 102 may also be configured to generate a 3D model based on the acquired image / depth information. Reference is made below to Figure 1 Explain the process in more detail.

[0023] The mobile application server 104 includes any computing device capable of generating a data stream enhanced with virtual fitting data for a user according to the techniques described herein. To generate the enhanced data stream, the mobile application server 104 may receive user data 106 from the user device 102. It should be noted that although the mobile application server 104 may receive the user data 106 while receiving a request to generate virtual fitting data, the mobile application server 104 may also receive the user data 106 prior to and independently of any request to generate virtual fitting data. For example, the mobile application server 104 may receive the user data 106 during a registration phase when a user establishes an account in the mobile application server 104.

[0024] The request for virtual fitting data may refer to stream data 108. Stream data 108 may be a streaming video (e.g., a live stream) or other suitable dynamic media content. Stream data 108 may be shown as at least one presenter 110 and at least one product 112. Mobile application server 104 may obtain an identifier (product identifier 114) of at least one product 112 and data related to the presenter's posture (posture data 116) from stream data 108. In some embodiments, one or more of product identifier 114 or posture data 116 may be associated with stream data 108 via metadata attached to stream data 108. In some embodiments, one or more machine vision techniques may be used to determine one or more of product identifier 114 and / or posture data 116 from an image within stream data 108.

[0025] The mobile application server 104 may include or access object model data 118, from which product data 120 may be obtained to fulfill the request. The object model data 118 may include any computer-readable storage medium having one or more 3D models stored thereon. For example, the object model data 118 may be a database maintained by the mobile application server 104 or another server. The 3D models stored in the object model data 118 may represent products that can be worn by a user, such as clothing (e.g., clothes) or accessories. In some embodiments, the object model data 118 may store 3D models of multiple versions (e.g., different sizes and / or styles) of a product. When a product identifier 114 for a particular product is received, the mobile application server 104 retrieves the product data 120 from the object model data 118, which includes the 3D model associated with the particular product.

[0026] The mobile application server 104 can be configured to combine the user data 106 and the product data 120 to generate a try-on image for the user. The mobile application server 104 can also pose the try-on image according to the posture data 116. Once the try-on image is generated, the mobile application server 104 can enhance the stream data 108 using the try-on image to generate enhanced stream data 122. Once the enhanced stream data 122 is generated, the enhanced stream data 122 can be sent back to the user device 102, and the enhanced stream data 122 can be rendered on a display for viewing by the user.

[0027] For the sake of clarity, Figure 1 A certain number of components are shown in FIG. However, it should be understood that the number of each component in the embodiments of the present invention can exceed one. In addition, some embodiments of the present invention may include less than or more than Figure 1 All components shown. In addition, Figure 1 The components in the system may communicate using any suitable communication protocol over any suitable communication medium, including the Internet.

[0028] Figure 2 The architecture of a system for augmenting streaming data with virtual fitting information according to at least some embodiments is shown. Figure 2 In the embodiment, the user device 202 can communicate with multiple other components including at least a mobile application server 204. The mobile application server 204 can perform at least a portion of the processing functions required by the mobile application installed on the user device. The user device 202 and the mobile application server 204 can be reference Figure 1 Examples of user device 102 and mobile application server 104 are described separately.

[0029] The user device 202 may be any suitable electronic device having at least some of the functions described in the present invention. Specifically, the user device 202 may be any electronic device capable of capturing user data and / or presenting an enhanced data stream on a display. In some embodiments, the user device may be able to establish a communication session with another electronic device (e.g., mobile application server 204) and send / receive data to / from the electronic device. The user device has the ability to download and / or execute mobile applications. The user device includes a mobile communication device as well as a personal computer and a thin client device. In some embodiments, the user device includes any portable electronic device having basic functions related to communication. For example, the user device may be a smart phone, a personal data assistant (PDA) or any other suitable handheld device. The user device may be implemented as a self-contained unit having various components (e.g., input sensors, one or more processors, memory, etc.) integrated into the user device. The reference to the "output" of a component or the "output" of a sensor in the present invention does not necessarily mean that the output is sent outside the user device. The output of various components may be retained in the self-contained unit defining the user device.

[0030] In one illustrative configuration, the user device 202 may include at least one memory 206 and one or more processing units (or processors) 208. The processor 208 may be appropriately implemented as hardware, computer executable instructions, firmware, or a combination thereof. The computer executable instructions or firmware implementation of the processor 208 may include computer executable instructions or machine executable instructions written in any suitable programming language for performing the various functions described. The user device 202 may also include one or more input sensors 210 for receiving user input and / or environmental input. There may be various input sensors 210 capable of detecting user input or environmental input, such as accelerometers, camera devices, depth sensors, microphones, global positioning system (e.g., GPS) receivers, etc. The one or more input sensors 210 may include a ranging camera device (e.g., a depth sensor) capable of generating a depth image and a camera device for acquiring image information.

[0031] For the purposes of the present invention, a rangefinder camera (e.g., a depth sensor) may be any device for identifying the distance or range of one or more objects from the rangefinder camera. In some embodiments, the rangefinder camera may generate a depth image (or depth map), and the pixel value on the image (or depth map) corresponds to the detection distance of the pixel. The pixel value may be obtained directly in physical units (e.g., meters). In at least some embodiments of the present invention, the user device may employ a rangefinder camera that works using structured light. In a rangefinder camera that works using structured light, a projector projects light onto one or more objects in a structured pattern. The light may be outside the visible light range (e.g., infrared or ultraviolet). The rangefinder camera has one or more camera devices that are used to acquire an image of an object having a reflection pattern. Distance information may then be generated based on the distortion in the detected pattern. It should be noted that although the present invention focuses on the use of rangefinder cameras using structured light, any suitable type of rangefinder camera, including those that work using stereo triangulation, light sheet triangulation, time of flight, interferometry, coded aperture, or any other technology suitable for distance detection, may be applicable to the described system.

[0032] The memory 206 stores program instructions that can be loaded and executed on the processor 208, and stores data generated during the execution of these programs. Depending on the configuration and type of the user device 202, the memory 206 can be volatile (e.g., random access memory (RAM)) and / or non-volatile (e.g., read-only memory (ROM), flash memory, etc.). The user device 202 also includes additional memory 212, such as removable memory or non-removable memory, including but not limited to magnetic memory, optical disk and / or tape memory. The disk drive and its associated computer-readable medium can provide non-volatile storage of computer-readable instructions, data structures, program modules and other data for the computing device. In some embodiments, the memory 206 may include a variety of different types of memory, such as static random access memory (SRAM), dynamic random access memory (DRAM) or ROM. The contents of the memory 206 are described in detail below. The memory 206 includes an operating system 214 and one or more applications or services for implementing the features disclosed in the present invention, and the application or service includes at least a mobile application 216. The memory 206 also includes application data 218, and the memory 206 provides information generated and / or used by the mobile application 216. In some embodiments, application data 218 may be stored in a database.

[0033] For purposes of the present invention, a mobile application may be any set of computer executable instructions that are installed and executed on a user device 202. The mobile application may be installed on the user device by the user device manufacturer or by another entity. In some embodiments, the mobile application 216 may enable the user device to establish a session with a mobile application server 204, which provides backend support for the mobile application 216. The mobile application server 204 may maintain account information associated with a particular user device and / or user. In some embodiments, the user may be required to log in to the mobile application to access functionality provided by the mobile application 216.

[0034] According to at least some embodiments, the mobile application 216 is configured to provide user information to the mobile application server 204 and present to the user information received from the mobile application server 204. More specifically, the mobile application 216 is configured to obtain measurement data of the user and submit the measurement data to the mobile application server 204 in association with a request for stream data enhanced with virtual fitting data. In some embodiments, the mobile application 216 may also receive an indication of a data stream enhanced with virtual fitting data.

[0035] According to at least some embodiments, the mobile application 216 can receive output from the input sensor 210 and generate a 3D model based on the output. For example, the mobile application 216 can receive depth information (e.g., a depth image image) from a depth sensor (e.g., a rangefinder camera), wherein the depth sensor can be, for example, a depth sensor mentioned in the previous description of the input sensor 210, and the mobile application 216 can also receive image information from the camera input sensor. Based on this information, the mobile application 216 can determine the edge of the object (e.g., the user) to be identified. For example, a sudden change in depth within the depth information can indicate the edge or outline of the object. In another example, the mobile application 216 can use one or more machine vision techniques and / or machine learning to identify the edge of the object. In this example, the mobile application 216 can receive image information from the camera input sensor 210, and can identify potential objects within the image information based on differences in color or texture data detected within the image or based on learned patterns. In some embodiments, the mobile application 216 may cause the user device 202 to send output obtained from the input sensor 210 to the mobile application server 204, which may then perform one or more object recognition techniques on the output to generate a 3D model of the object.

[0036] User device 202 also includes a communication interface 220 that enables user device 202 to communicate with any other suitable electronic device. In some embodiments, communication interface 220 may enable user device 202 to communicate with other electronic devices on a network (e.g., a private network). For example, user device 202 may include a Bluetooth® that allows it to communicate with another electronic device. TM (BLUETOOTH TM ) wireless communication module. The user device 202 also includes an input / output (I / O) device and / or port 222, for example, for connecting to a keyboard, a mouse, a pen, a voice input device, a touch input device, a display, a speaker, a printer, etc.

[0037] In some embodiments, user device 202 can communicate with mobile application server 204 via a communication network. The communication network can include any one or a combination of multiple different types of networks such as cable networks, the Internet, wireless networks, cellular networks, and other private and / or public networks. In addition, the communication network includes a variety of different networks. For example, user device 202 can communicate with a wireless router via a wireless local area network (WLAN), and the wireless router can then route the communication to mobile application server 204 via a public network (e.g., the Internet).

[0038] The mobile application server 204 can be any computing device or multiple computing devices for performing one or more calculations for the mobile application 216 on the user device 202. In some embodiments, the mobile application 216 can communicate with the mobile application server 204 periodically. For example, the mobile application 216 can receive updates, push notifications or other instructions from the mobile application server 204. In some embodiments, the mobile application 216 and the mobile application server 204 can use proprietary encryption and / or decryption schemes to protect the communication between the two. In some embodiments, the mobile application server 204 can be executed by one or more virtual machines implemented in a hosted computing environment. The hosted computing environment includes one or more computing resources that are quickly provided and released, and the computing resources may include computing, networking and / or storage devices. The hosted computing environment may also be referred to as a cloud computing environment.

[0039] In one illustrative configuration, the mobile application server 204 may include at least one memory 224 and one or more processing units (or processors) 226. The processor 226 may be suitably implemented as hardware, computer executable instructions, firmware, or a combination thereof. The computer executable instructions or firmware implementation of the processor 226 may include computer executable instructions or machine executable instructions written in any suitable programming language for performing the various functions described.

[0040] The memory 224 can store program instructions that can be loaded and executed on the processor 226, as well as data generated during the execution of these programs. Depending on the configuration and type of the mobile application server 204, the memory 224 can be volatile (e.g., RAM) and / or non-volatile (e.g., ROM, flash memory, etc.). The mobile application server 204 also includes additional memory 228, such as removable memory or non-removable memory, including but not limited to magnetic memory, optical disk and / or tape memory. The disk drive and its associated computer-readable medium can provide non-volatile storage of computer-readable instructions, data structures, program modules and other data for the computing device. In some embodiments, the memory 224 can include a variety of different types of memory, such as SRAM, DRAM or ROM. The contents of the memory 224 are described in detail below. The memory 224 includes an operating system 230 and one or more applications or services for implementing the features disclosed in the present invention, the application or service at least including a module for trying on a product 3D model on a user 3D model (try-on module 232) and / or a module for determining a posture and applying the posture to the product 3D model and the user 3D model (gesture module 234). The memory 224 also includes account data 236 that provides information associated with a user account maintained by the system; user model data 238 that maintains a 3D model associated with each user of the account; and / or object model data 240 that maintains 3D models associated with a plurality of objects (products). In some embodiments, one or more of the account data 236, user model data 238, or object model data 240 may be stored in a database. In some embodiments, the object model data 240 may be an electronic catalog that includes data related to objects that may be sold from a resource provider (e.g., a retailer or other suitable merchant).

[0041] Memory 224 and additional memory 228 are examples of computer-readable storage media, which may be removable or non-removable. For example, computer-readable storage media may include volatile or non-volatile, removable or non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. The term "module" as used in the present invention refers to a programming module executed by a computing system (e.g., a processor) installed on and / or executed from the mobile application server 204. The mobile application server 204 also includes a communication connection 242, which allows the mobile application server 204 to communicate with a stored database, another computing device or server, a user terminal and / or other components of the system. The mobile application server 204 also includes an input / output (I / O) device and / or port 244, for example, for realizing connections with a keyboard, a mouse, a pen, a voice input device, a touch input device, a display, a speaker, a printer, etc.

[0042] The contents of the memory 224 are described in detail below. The memory 224 includes a try-on module 232 , a gesture module 234 , a database containing account data 236 , a database containing user model data 238 , and / or a database containing object model data 240 .

[0043] In some embodiments, the try-on module 232 may be configured to apply deformations to the product 3D model together with the processor 226 so as to try it on the user 3D model. The try-on module 232 may access one or more rules that describe how to deform (e.g., stretch and / or bend) a specific product type (e.g., shirt, pants, etc.) so as to try it on the user model. In order to try the product 3D model on the user 3D model, the try-on module 232 may align certain parts of the product 3D model to specific parts of the user 3D model. For example, the 3D model of the shirt may be placed so that the sleeves of the 3D model of the shirt surround the arms of the user 3D model. In addition, the 3D model of the shirt may also be placed so that the collar of the 3D model of the shirt surrounds the neck of the user 3D model. Then, the rest of the 3D model of the shirt may be deformed by stretching and bending portions of the 3D model of the shirt so that the inside of the 3D model of the shirt is outside the 3D model of the user or along the outside of the 3D model of the user.

[0044] In some embodiments, the posture module 234 may be configured to identify the posture of the presenter (i.e., human body) in the streaming data together with the processor 226, and apply the posture to the combination of the product 3D model and the user 3D model generated by the try-on module 232. This includes using one or more posture estimation techniques to determine the current posture of the presenter in the data stream. For example, the posture module 234 can use machine learning to determine the posture of the presenter in the data stream. Those skilled in the art will appreciate that a variety of suitable posture estimation techniques can be used. In some embodiments, the posture module 234 can apply the determined posture to the user model (the product model has been tried on the user model). This includes repositioning one or more accessories or body parts of the user model until the determined posture is achieved. In some embodiments, the posture module 234 can monitor the posture of the presenter in the streaming data, and can adjust the posture of the user model to match the posture of the presenter when a change in the posture of the presenter is detected.

[0045] After adjusting the posture of the combined user model and product model, the posture module 234 can render the combined user model and product model. In some embodiments, the combined user model and product model can be rendered in a small window, which is then placed in an inconspicuous position in the streaming data. For example, in the case where the streaming data is a video, the combined user model and product model can be rendered in a window in the lower corner of the video. This rendering can make the user imagine the product wearing him / herself. It should be noted that although the posture module 234 and the try-on module 232 are described with reference to the mobile application server 204, the functions described as being performed by one or more modules can be performed by a mobile application on the user device 202.

[0046] In some embodiments, each object entry within object model database 240 may be associated with a 3D model of the object. In these embodiments, the 3D model may be combined with a second 3D model of the user and provided to mobile application 216, so that user device 202 displays the combination of the 3D models on a display of the user device as an enhancement to the streaming data. As the pose of the presenter within the streaming data is updated, mobile application 216 may dynamically update the pose of the combination of the 3D models on the display of the user device.

[0047] Figure 3A simplified flow chart of a method for presenting a data stream enhanced with virtual fitting data provided by an embodiment of the present invention is shown. The process is introduced in conjunction with a computer system as an example of a computer system introduced in the present invention. Some or all of the operations of the process can be implemented by specific hardware on the computer system and / or as computer-readable instructions stored on a non-temporary computer-readable medium of the computer system. As stored, the computer-readable instructions represent a programmable module including a code that can be executed by a computer system processor. The execution of such instructions configures the computer system to perform the corresponding operation. Each programmable module and the processor together represent a device for performing the corresponding operation. Although these operations are shown in a specific order, it should be understood that the specific order is not necessary, and one or more operations can be omitted, skipped and / or reordered.

[0048] At the beginning of process 300, one or more products 302 are scanned to generate product model data 304. The generated product model 304 is a 3D virtual representation of the product 302, and the product model 304 can be generated by scanning the product 302 from multiple perspectives using a camera and / or a depth sensor. In step 1 of process 300, the multiple generated product models 304 are provided to the mobile application server 204 to be stored in the object model data 240. The product model 304 can be generated by multiple different entities. For example, the product model of a specific product can be generated by the manufacturer of the product.

[0049] In addition, in step 2 of process 300, the user is scanned using a camera and / or a depth sensor installed on the user device 202 to generate user model data 308. Figure 4 Some example techniques for generating an object (e.g., user) model are explained in more detail. In step 3 of process 300, user model data 308 is sent to mobile application server 204 for storage in user model data 238. In some cases, user model data 308 may be stored in association with an account maintained for the scanned user.

[0050] The mobile application server 204 receives a request from the user to use the streaming data 306. In step 4 of process 300, upon receiving the request to use the streaming data 306, the mobile application server 204 retrieves the streaming data 306 from its location. In some embodiments, the streaming data 306 may be maintained by the mobile application server 204. In some embodiments, the streaming data 306 may be maintained by an entity separate from the mobile application server 204. For example, a user may request through the mobile application server 204 to watch a video hosted by YOUTUBE.com. TMThe mobile application server 204 provides support for a mobile application installed on a user's mobile device. In this example, the user can provide a uniform resource locator (URL) or other identifier of the video. The mobile application server 204 can then retrieve the video file by accessing the URL. Once the streaming data 306 is retrieved, the mobile application server 204 identifies one or more related products and the poses of the presenter. As described elsewhere, the above-mentioned Figure 2 The described fitting module 232 and / or posture module 234 are used to complete the task.

[0051] The steps performed by the posture module 234 are described below. In step 5 of process 300, process 300 includes determining the posture of a presenter in the stream data 306. This includes first identifying the presenter in the stream data (e.g., using one or more machine vision techniques), and then estimating the posture of the presenter using any suitable posture estimation technique. Those skilled in the art will appreciate that a variety of suitable techniques can be used. In general, the posture of an object indicates the position and orientation of the object. For a speaker, the estimated posture includes a record of the position and orientation of various body parts or joints of the presenter.

[0052] After estimating the presenter's posture, the posture module 234 applies the posture to the user model. To do this, at step 6 of process 300, the posture module 234 retrieves the user model. The user model is a three-dimensional model representing a person, which is stored in the user model data 238 in association with the person or an account linked to the person. When a request for usage stream data 306 about a particular user is received, the posture module 234 can retrieve the user model associated with the person from the user model data 238. Once retrieved, the posture module 234 applies the presenter's estimated posture to the retrieved user model. To do this, the posture module 234 repositions the various body parts of the user model to match the record of the position and orientation of the various body parts of the presenter. Then, at step 7 of process 300, the posed user model is provided to the try-on module 232.

[0053] The steps performed by the try-on module 232 are described below. In step 8 of process 300, process 300 includes retrieving one or more product models. First, the try-on module 232 identifies one or more products associated with the stream data 306. In some embodiments, the stream data 306 may include an indication of one or more products. For example, the stream data 306 may have additional metadata indicating a stock keeping unit (SKU) or other product identifier associated with the stream data 306. In some embodiments, machine vision techniques (e.g., object recognition) may be used to identify one or more products from the stream data 306. For example, the try-on module 232 may identify a specific product (e.g., a shirt or pants) worn by a presenter in the stream data 306 by comparing the visual attributes of the product with the attributes associated with multiple products stored in an electronic catalog maintained by the mobile application server 204. In this example, the try-on module 232 may identify the product in the electronic catalog that best matches the product worn by the presenter. Once the try-on module 232 identifies one or more products associated with the stream data 306, the product models of these products are retrieved from the object model data.

[0054] In step 9 of process 300, one or more product models obtained in step 8 are tried on the posed user model provided in step 7. The product model is tried on the posed user model by adjusting a set of parameters that control the deformation of one or more regions of interest of the product model until the product model matches the user model. The set of parameters can be defined as a set of measurements, such as the displacement of each vertex of the product model. This process can be described as an optimization process, in which several different optimization algorithms can be used to find the best set of parameters that minimize one or more cost functions. The cost function can be defined as the number of penetrations between the meshes of the two 3D models, the average distance between the body mesh vertices and the clothing mesh vertices, etc. U.S. patent application number 62 / 987,196, entitled "SYSTEM AND METHOD FOR VIRTUAL FITTING", describes further examples of techniques for rendering product models tried on user models in more detail, and the entire contents of the U.S. patent application are hereby incorporated by reference into the present invention for all purposes.

[0055] Once the product models are tried on the user model, the models are rendered at step 10. Rendering is the process of giving a 3D model a solid appearance using shading and color. Those skilled in the art will recognize that there are a variety of suitable techniques for rendering the product models tried on the posed user model. In some embodiments, the stream data 306 is enhanced with the rendered model. For example, the rendered model can be placed as enhanced visual data in a small window within the stream data 306, so that a viewer of the stream data 306 (e.g., a user) can view the rendered model while viewing the stream data 306.

[0056] Then, at step 11 of process 300, the rendered model (e.g., enhanced streaming data) is provided to user device 202. Upon receiving the rendered model, user device 202 may present the rendered model to the user. For example, user device 202 may play the enhanced streaming data via a media player application.

[0057] In some embodiments, additional processing may be performed when using the enhanced streaming data 306. For example, the user device 202 that is presenting the enhanced streaming data 306 may collect image information of the user who is viewing the enhanced streaming data 306 through a front-facing camera installed on the user device 202. In this example, the user's facial data may be extracted from the image information and overlaid onto the rendered user model, thereby providing the user's facial and facial expression data to the user model.

[0058] It should be understood that Figure 3 The specific steps shown provide a specific method for presenting a data stream enhanced with virtual fitting data according to an embodiment of the present invention. As described above, according to alternative embodiments, the steps may also be performed in other orders. For example, alternative embodiments of the present invention may perform the above steps in a different order. In addition, Figure 3 The various steps shown may include multiple sub-steps, which may be performed in various orders suitable for the various steps. In addition, steps may be added or deleted according to specific applications. Many variations, modifications, and alternatives may be obtained by those skilled in the art.

[0059] Figure 4 An illustrative example of a technique for obtaining a 3D model using sensor data according to at least some embodiments is shown. According to at least some embodiments, sensor data 402 can be obtained from one or more input sensors installed on a user device. The captured sensor data 402 includes image information 404 captured by a camera device and depth map information 406 captured by a depth sensor.

[0060] As described above, sensor data 402 includes image information 404. One or more image processing techniques may be used on image information 404 to identify one or more objects within image information 404. For example, edge detection may be used to identify region 408 within image information 404 that includes objects. To this end, discontinuities in brightness, color, and / or texture may be identified on the image to detect the edges of various objects within the image. Region 408 shows an illustrative example image of a chair that highlights such discontinuities.

[0061] As described above, sensor data 402 includes depth information 406. In depth information 406, each pixel may be assigned a value that represents the distance between the user device and a particular point corresponding to the pixel location. Depth information 406 may be analyzed to detect sudden changes in depth within depth information 406. For example, a sudden change in distance may indicate an edge or boundary of an object within depth information 406.

[0062] In some embodiments, the sensor data 402 includes both image information 404 and depth information 406. In at least some of these embodiments, an object may be first identified in either the image information 404 or the depth information 406, and various attributes of the object may be determined from the other information. For example, edge detection techniques may be used to identify a region 408 within the image information 404 that includes an object. The region 408 may then be mapped to a corresponding region 410 in the depth information to determine depth information (e.g., a point cloud) for the identified object. In another example, a region 410 may first be identified within the depth information 406 that includes an object. In this example, the region 410 may then be mapped to a corresponding region 408 in the image information to determine an appearance attribute (e.g., a color or texture value) of the identified object.

[0063] In some embodiments, various attributes of the object identified in the sensor data 402 (e.g., color, texture, point cloud data, object edges) can be used as input to a machine learning module to identify or generate a 3D model 412 that matches the identified object. In some embodiments, a point cloud of the object can be generated from the depth information and / or image information and compared to the point cloud data stored in the database to identify the best matching 3D model. Alternatively, a 3D model of an object (e.g., a user or a product) can be generated using the sensor data 402. To do this, a mesh can be created using the point cloud data obtained from the region 410 of the depth information 406. The system can then map the appearance data from the region corresponding to the region 410 in the image information 404 to the mesh to generate a basic 3D model. Although specific techniques are described, it should be noted that there are many techniques for identifying specific objects from sensor output.

[0064] As described elsewhere, by a user device (e.g., Figure 1 Sensor data captured by a user device 102 of a user may be used to generate a user 3D model using the above techniques. The user 3D model may then be provided to the mobile application server as user data. In some embodiments, the sensor data may be used to generate a 3D model of a product, which may be stored in user model data 238. For example, a user who wishes to sell a product may capture sensor data related to the product from a user device. The user's user device may then generate a 3D model in the manner described above and provide the 3D model to the mobile application server.

[0065] Figure 5 An example of a graphical user interface (GUI) is shown for demonstrating some example features that may be implemented according to an embodiment of the present invention. Figure 5 In the example user device 502 is shown as having a display screen on which visual data can be presented. The user device 502 is the same as the one described above with reference to Figure 2 An example of a user device 202 is described.

[0066] like Figure 5 As shown, a GUI of a software application (e.g., a media viewer application) installed on a user device 502 may be used to present streaming data 504. Streaming data 504 includes at least at least one presenter 506 and a product 508, wherein the presenter is the person shown in the streaming data 506. Product 508 may be worn or otherwise presented by presenter 506 within streaming data 504.

[0067] As described elsewhere, a posed and tried-on model 510 may be presented with the streaming data 504. For example, the model 510 may be presented within a separate window 512 that is positioned to minimize any obstruction to viewing of the streaming data 504, sometimes referred to as a picture-in-picture. The model 510 includes a user model representing the current viewer of the streaming data 504, the user model having posed in a manner similar to the presenter 508 and having tried on a product model representing the product 508.

[0068] Figure 6 A flow chart of a process for presenting virtual fitting data to a user, provided by at least some embodiments, is shown. Figure 6 The process 600 shown in FIG. 6 may be performed by a user device (eg, Figure 2 The user device 202) communicates with a mobile application server (eg, Figure 2 The mobile application server 204) executes.

[0069] At 602, process 600 includes receiving an indication of media content being used by a user. For example, an indication may be received that the user is watching a streaming video, where the streaming video is a type of media content. The indicated media content may include a display of a presenter, where the presenter is a person other than the user. The indicated media content may also include a description of a product presented by the presenter. For example, the product may be clothing worn by the presenter in the media content.

[0070] At 604, process 600 includes identifying a product associated with the media content. In some embodiments, the product associated with the media content is identified by an identifier associated with the product contained in metadata of the media content. In one example, the identifier associated with the product is a SKU number. In some embodiments, the product associated with the media content is identified by object recognition.

[0071] At 606, process 600 includes obtaining a first 3D model representing the product. To this end, the first 3D model having object model data (e.g., Figure 2 The 3D model associated with the product identified in 604 may be retrieved from a database of object model data 240. In some embodiments, an appropriate size and / or style of the product may be selected based on stored information about the user who is viewing the media content.

[0072] At 608, process 600 includes obtaining a second 3D model representing the user. In some embodiments, the user model can be stored in association with one or more accounts. In these embodiments, the second 3D model can be identified and retrieved by being stored in association with the account used to view the media content. In some embodiments, the second 3D model representing the user can be received from a user device that is being used to view the media content.

[0073] At 610, process 600 includes determining a presentation pose based on the media content. The presentation pose is determined as the presenter's current pose within the media content. Any suitable pose estimation technique may be used to accomplish this. The determined presentation pose includes indications of various parts (e.g., body parts) of the user model and their respective positions and orientations.

[0074] At 612, process 600 includes applying the demonstration gesture to the second 3D model. To this end, the positions and orientations of various parts (eg, body parts) of the second 3D model may be adjusted so that they match corresponding positions and orientations in the demonstration gesture data.

[0075] At 614, process 600 includes generating a third 3D model by fitting the second 3D model through the first 3D model. Here, the first 3D model is deformed to minimize a distance between the first 3D model and the second 3D model.

[0076] At 616, process 600 includes presenting the third 3D model to the user. This involves rendering the third 3D model and providing the third 3D model to the user device that is presenting the media content. The third 3D model is presented together with the media content. For example, the media content can be enhanced to include the third 3D model (e.g., in a separate window within the media content).

[0077] It should be understood that according to the embodiments of the present invention, Figure 6 The specific steps shown provide a specific method for presenting virtual fitting data to a user. As described above, other step sequences may also be performed according to alternative embodiments. For example, alternative embodiments of the present invention may perform the above steps in a different order. In addition, Figure 6 The various steps shown may include multiple sub-steps, which may be performed in various orders suitable for the various steps. In addition, steps may be added or deleted according to specific applications. Many variations, modifications, and alternatives may be obtained by those skilled in the art.

[0078] Figure 7 An example of computer system components provided by some embodiments is shown. Computer system 700 is an example of a computer system described in the present disclosure. Although these components are shown as belonging to the same computer system 700, the computer system 700 may also be distributed.

[0079] The computing system 700 includes at least a processor 702, a memory 704, a storage device 706, an input / output (I / O) peripheral device 708, a communication peripheral device 710, and an interface bus 712. The interface bus 712 can be used to communicate, send, and transmit data, control, and commands between the various components of the computing system 700. The memory 704 and the storage device 706 may include computer-readable storage media, such as RAM, ROM, electrically erasable programmable read-only memory (EEPROM), hard disk drives, CD-ROMs, optical storage devices, magnetic storage devices, electronic non-volatile computer storage (e.g., memory), and other tangible storage media. Any such computer-readable storage medium can be used to store instructions or program codes for implementing various aspects of the present disclosure. The memory 704 and the storage device 706 may also include computer-readable signal media. Computer-readable signal media include propagated data signals containing computer-readable program codes. Such propagated signals take any of a variety of forms, including but not limited to electromagnetic, optical, or any combination thereof. Computer readable signal media include any computer readable media that is not a computer readable storage media and that can communicate, propagate, or transport a program for use in conjunction with computing system 700 .

[0080] In addition, the memory 704 package may include an operating system, programs and applications. The processor 702 may be used to execute stored instructions and include, for example, a logic processing unit, a microprocessor, a digital signal processor and other processors. The memory 704 and / or the processor 702 may be virtualized and may be hosted in another computing system such as a cloud network or a data center. The input / output peripherals 708 may include a user interface, such as a keyboard, a screen (e.g., a touch screen), a microphone, a speaker, other input / output devices, and computing components such as a graphics processing unit, a serial port, a parallel port, a universal serial bus and other input / output peripherals. The input / output peripherals 708 are connected to the processor 702 by any port coupled to the interface bus 712. The communication peripherals 710 may be used to facilitate communication between the computing system 700 and other computing devices through a communication network, and include, for example, a network interface controller, a modem, a wireless and wired interface card, an antenna and other communication peripherals.

[0081] Figure 8 A block diagram of an apparatus for presenting virtual fitting data to a user, provided by at least some embodiments, is shown. Figure 8 The apparatus 800 shown in FIG. 8 may be implemented as a user device (eg, Figure 2The user device 202) communicates with a mobile application server (eg, Figure 2 Mobile application server 204).

[0082] The apparatus 800 includes a receiving module 802 configured to receive an indication of media content that a user is viewing. For example, an indication may be received that a user is viewing a streaming video, where streaming video is a type of media content. The indicated media content may include a description of a presenter, where the presenter is a person other than the user. The indicated media content may also include a description of a product presented by the presenter. For example, the product may be clothing worn by the presenter in the media content.

[0083] The device 800 also includes an identification module 804, which is configured to identify a product associated with the media content. In some embodiments, the product associated with the media content is identified by an identifier, and the identifier is associated with the product contained in the metadata of the media content. In one example, the identifier associated with the product is a stock keeping unit (SKU) number. In some embodiments, the product associated with the media content is identified by object recognition.

[0084] The apparatus 800 further comprises an acquisition module 806, which is configured to acquire a first three-dimensional (3D) model representing the product. To this end, the object model data (e.g., Figure 2 The 3D model associated with the product identified in 604 may be retrieved from a database of object model data 240. In some embodiments, an appropriate size and / or style of the product may be selected based on stored information about the user who is viewing the media content.

[0085] The acquisition module 806 is also configured to acquire a second 3D model representing the user. In some embodiments, the user model can be stored in association with one or more accounts. In these embodiments, the second 3D model can be identified and retrieved by storing it in association with the account used to view the media content. In some embodiments, the second 3D model representing the user can be received from a user device that is being used to view the media content.

[0086] The apparatus 800 also includes a determination module 808, which is configured to determine a demonstration gesture based on the media content. The demonstration gesture is determined as the current gesture of the presenter in the media content. Any suitable gesture estimation technology can be used to complete this. The determined demonstration gesture may include indications of various parts (e.g., body parts) of the user model and their respective positions and orientations.

[0087] The device 800 also includes an application module 810, which is configured to apply the demonstration posture to the second 3D model; to this end, the position and orientation of various parts (e.g., body parts) of the second 3D model can be adjusted to match the corresponding position and orientation in the demonstration posture data.

[0088] The apparatus 800 further includes a generating module 812 configured to generate a third 3D model by making the second 3D model try on the first 3D model. Here, the first 3D model is deformed to minimize the distance between the first 3D model and the second 3D model.

[0089] The apparatus 800 also includes a rendering module 814, which is configured to present the third 3D model to the user. This involves rendering the third 3D model and providing the third 3D model to the user device that is presenting the media content. The third 3D model is presented together with the media content. For example, the media content can be enhanced to include the third 3D model (e.g., in a separate window within the media content).

[0090] Although the subject matter has been described in detail with respect to its specific embodiments, it should be understood that those skilled in the art, after obtaining an understanding of the foregoing, can easily generate changes, variations and equivalents to these embodiments. Therefore, it should be understood that the present disclosure is presented for the purpose of illustration rather than limitation, and does not exclude the inclusion of such modifications, variations and / or additions to the subject matter that are obvious to a person of ordinary skill in the art. In fact, the methods and systems described in the present disclosure can be implemented in a variety of other forms; in addition, various omissions, substitutions and changes can be made to the form of the methods and systems described in the present disclosure without departing from the spirit of the present disclosure. The attached claims and their equivalents are intended to cover such forms or modifications that fall within the scope and spirit of the present disclosure.

[0091] Unless expressly stated otherwise, it should be understood that terms such as "process," "compute," "calculate," "determine," and "identify" are used throughout the discussion of this specification to refer to the actions or processes of a computing device (e.g., one or more computers or similar electronic computing devices) that manipulates or transforms data represented as physical electronic or magnetic quantities in the memory, registers, or other information storage devices, transmission devices, or display devices of a computing platform.

[0092] The one or more systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device may include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multi-purpose microprocessor-based computer systems that access stored software that programs or configures the computing system from a general-purpose computing device to a dedicated computing device that implements one or more embodiments of the present subject matter. Any suitable programming, scripting, or other type of language or combination of languages ​​may be used to implement the teachings contained herein in software for programming or configuring a computing device.

[0093] Embodiments of the methods disclosed herein may be performed in the operation of such a computing device. The order of the blocks presented in the above examples may be changed, for example, the blocks may be reordered, combined and / or decomposed into sub-blocks. Certain blocks or processes may be performed in parallel.

[0094] Conditional language used herein, such as "can," "may," "might," "could," "for example," and the like, unless expressly stated otherwise or otherwise understood in the context of use, is generally intended to convey that certain examples include and other examples do not include certain features, elements, and / or steps. Thus, such conditional language generally does not imply that one or more examples require the features, elements, and / or steps in any way, or that one or more examples must include logic for deciding, with or without author input or prompting, whether such features, elements, and / or steps are included or will be performed in any particular example.

[0095] The terms "include", "comprises", "have", etc. are synonymous and are used inclusively in an open manner and do not exclude other elements, features, actions, operations, etc. In addition, the term "or" is used with its inclusiveness (not exclusiveness), so that when used, for example, to connect a list of elements, the term "or" represents one, some or all of the elements in the list. "Applicable to" or "for" used herein refers to open and inclusive language, and does not exclude devices that are applicable to or used to perform additional tasks or steps. In addition, the use of "based on" means open and inclusive, because the process, step, calculation or other action "based on" one or more listed conditions or values ​​may actually be based on additional conditions or values ​​outside the list. Similarly, the use of "based at least in part on" means open and inclusive, because the process, step, calculation or other action "based at least in part on" one or more listed conditions or values ​​may actually be based on additional conditions or values ​​outside the list. The titles, lists and numbers included herein are only for ease of explanation and are not meant to be limiting.

[0096] The various features and processes described above can be used independently of each other, or can be used in combination in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. In addition, some method or process blocks can be omitted in some embodiments. The methods and processes described herein are also not limited to any particular order, and the blocks or states related thereto can be performed in other appropriate orders. For example, the described blocks or states can be performed in a different order than that specifically disclosed, or multiple blocks or states can be combined in a single block or state. The example blocks or states can be performed serially, in parallel, or in some other way. Blocks or states can be added to or deleted from the disclosed examples. Similarly, the example systems and components described herein can be configured to be different from those described. For example, elements can be added, removed, or rearranged compared to the disclosed examples.

Claims

1. A method for virtual fitting, comprising: Receiving an indication of media content being viewed by a user; the media content comprising streaming video; identifying a product associated with the media content; acquiring a first 3D model representing the product; Acquire a second 3D model representing the user; determining a presentation gesture according to the media content; applying the demonstration pose to the second 3D model; Generate a third 3D model by making the second 3D model try on the first 3D model, so as to enhance the media content; and presenting the third 3D model to the user in the demonstration posture; dynamically updating a pose of the third 3D model as the pose of the presenter of the media content is updated; The step of generating a third 3D model by making the second 3D model try on the first 3D model comprises: fitting the first 3D model onto the second 3D model by adjusting parameters that control deformation of one or more regions of interest of the first 3D model until the first 3D model matches the second 3D model to generate the third 3D model; Wherein, applying the demonstration posture to the second 3D model comprises: The second 3D model is retrieved from the user model data, the body parts of the second 3D model are repositioned to match the positions and orientations of the body parts of the presenter, and the presentation gesture is applied to the second 3D model.

2. The method according to claim 1, wherein: The product associated with the media content is identified by an identifier associated with the product contained in metadata of the media content.

3. The method according to claim 2, wherein: The identifier associated with the product is a stock keeping unit (SKU) number.

4. The method according to claim 1, wherein: A first 3D model representing the product is obtained from a catalog of 3D models.

5. The method according to claim 1, wherein: The product is an article of clothing worn by a presenter within the media content, and wherein the presentation pose comprises a pose of the presenter.

6. The method according to claim 5, wherein: The presenters include a second user different from the user.

7. A system for virtual fitting, comprising: processor; as well as A memory including instructions that, when executed by the processor, cause the system to at least: Receiving an indication of media content being viewed by a user; the media content comprising streaming video; identifying a product associated with the media content; acquiring a first 3D model representing the product; Acquire a second 3D model representing the user; determining a presentation gesture according to the media content; applying the demonstration pose to a second 3D model; Generate a third 3D model by making the second 3D model try on the first 3D model, so as to enhance the media content; and presenting the third 3D model to the user in the demonstration posture; As the pose of the presenter of the media content is updated, dynamically updating the pose of the third 3D model; The step of generating a third 3D model by making the second 3D model try on the first 3D model comprises: fitting the first 3D model onto the second 3D model by adjusting parameters that control deformation of one or more regions of interest of the first 3D model until the first 3D model matches the second 3D model to generate the third 3D model; Wherein, applying the demonstration posture to the second 3D model comprises: The second 3D model is retrieved from the user model data, the body parts of the second 3D model are repositioned to match the positions and orientations of the body parts of the presenter, and the presentation gesture is applied to the second 3D model.

8. The system according to claim 7, wherein: The product associated with the media content is identified by an identifier associated with the product contained in metadata of the media content.

9. The system according to claim 8, wherein: The identifier associated with the product is a stock keeping unit (SKU) number.

10. The system according to claim 7, wherein: The first 3D model representing the product is obtained from a 3D model catalog.

11. The system according to claim 7, wherein: The product is an article of clothing worn by a presenter within the media content, and wherein the presentation pose comprises a pose of the presenter.

12. The system according to claim 11, wherein: The presenters include a second user different from the user.

13. A non-transitory computer-readable medium storing specific computer-executable instructions that, when executed by a processor, cause a computer system to at least: Receiving an indication of media content being viewed by a user; the media content comprising streaming video; identifying a product associated with the media content; acquiring a first 3D model representing the product; Acquire a second 3D model representing the user; determining a presentation gesture according to the media content; applying the demonstration pose to a second 3D model; Generate a third 3D model by making the second 3D model try on the first 3D model, so as to enhance the media content; as well as presenting the third 3D model to the user in the demonstration posture; dynamically updating a pose of the third 3D model as the pose of the presenter of the media content is updated; The step of generating a third 3D model by making the second 3D model try on the first 3D model comprises: fitting the first 3D model onto the second 3D model by adjusting parameters that control deformation of one or more regions of interest of the first 3D model until the first 3D model matches the second 3D model to generate the third 3D model; Wherein, applying the demonstration posture to the second 3D model comprises: The second 3D model is retrieved from the user model data, the body parts of the second 3D model are repositioned to match the positions and orientations of the body parts of the presenter, and the presentation gesture is applied to the second 3D model.

14. The non-transitory computer readable medium of claim 13, wherein: The product associated with the media content is identified by an identifier associated with the product contained in metadata of the media content.

15. The non-transitory computer readable medium of claim 14, wherein: The identifier associated with the product is a stock keeping unit (SKU) number.

16. The non-transitory computer readable medium of claim 13, wherein: A first 3D model representing the product is obtained from a catalog of 3D models.

17. The non-transitory computer readable medium of claim 13, wherein: The product is an article of clothing worn by a presenter within the media content, and wherein the presentation pose comprises a pose of the presenter.

18. A device for virtual fitting, comprising: a receiving module configured to receive an indication of media content being viewed by a user; The media content includes streaming video; an identification module configured to identify a product associated with the media content; An acquisition module, configured to acquire a first 3D model representing the product and a second 3D model representing the user; a determination module, configured to determine a presentation gesture according to the media content; an application module configured to apply the demonstration gesture to a second 3D model; A generating module, configured to generate a third 3D model by making the second 3D model try on the first 3D model, so as to enhance the media content; as well as a presentation module, configured to present the third 3D model to the user in the demonstration posture; dynamically updating a pose of the third 3D model as the pose of the presenter of the media content is updated; The generating module is specifically configured to fit the first 3D model onto the second 3D model by adjusting parameters, wherein the parameters control deformation of one or more regions of interest of the first 3D model until the first 3D model matches the second 3D model, so as to generate the third 3D model; The application module is specifically configured to retrieve the second 3D model from the user model data, reposition the various body parts of the second 3D model to match the position and orientation of the various body parts of the presenter, and apply the demonstration posture to the second 3D model.

19. The device according to claim 18, wherein: The product associated with the media content is identified by an identifier associated with the product contained in metadata of the media content.

20. The device according to claim 19, wherein The identifier associated with the product is a stock keeping unit (SKU) number.

21. The device according to claim 18, wherein The first 3D model representing the product is obtained from a 3D model catalog.

22. The device according to claim 18, wherein The product is an article of clothing worn by a presenter within the media content, and wherein the presentation pose comprises a pose of the presenter.

23. The device according to claim 22, wherein: The presenters include a second user different from the user.

Images