A method and system for generating a model of a three-dimensional item and a method of providing a virtual wardrobe

Abstract

The invention relates to a method and system for generating a model of a three-dimensional item, the method comprising: capturing multiple images of the item using a digital comprising a mobile telephone or a personal digital device; extracting data at the device from the multiple images captured by the device; using machine learning, generating a three-dimensional base image of the object.

Description

[0001] A Method and System for Generating a Model of a Three-Dimensional Item and a Method of Providing a Virtual Wardrobe

[0002] The present invention relates to a method and system for generating a model of a three-dimensional item, a virtual wardrobe and a method of providing a virtual wardrobe.

[0003] Online shopping is well known technology and relies on a user visiting a vendor’s website and identifying items of interest which can then be purchased. Such technology is old and well known and although for many types of purchase it works well, particularly if a user is purchasing an item which they have purchased before, a problem is encountered in the sale of items which require some form of fitting. Clothing is commonly purchased online in this very well-known manner, but if it does not fit well, which the user will only discover once they are in receipt of the physical item, it can be inconvenient and frustrating for the user to have to send it back or return it to the vendor in some other way.

[0004] To address this problem some vendors have implemented systems in which a three-dimensional render can be created to simulate how an item of clothing, of interest to a user, might appear when worn by the user. This increases the chance that the purchase when made, will not be regretted, or considered an error once the physical item of clothing is actually received by the user.

[0005] However, existing system for enabling a user to see what they might look like wearing an as yet unpurchased item of clothing are limited. The renders or representations that are produced, of a user wearing the item, are not accurate and so the problem still exists that a user ends up purchasing an item when there is a significant chance that they will not be satisfied with it when they actually receive it from the vendor.

[0006] Some attempts have been made to address this problem. For example, LIS2018005375 discloses an inventory capture system including a method and apparatus (i.e. , the inventory capture system) for creating and updating an inventory of clothing for a user. The inventory capture system may use voice and image recognition to capture an inventory of clothing and provide users the ability to enhance the captured details about an inventory of clothing with annotations. Moreover, the inventory capture system may provide a way to facilitate retailers and users to leverage the user's existing inventory of clothing and augment the user's inventory of clothing with shared, purchased and / or rented clothing.

[0007] Other systems have used augmented reality. US2024037858 discloses a system for providing an Augmented Reality (AR) experience. The system accesses, by a messaging application, an image depicting a real-world fashion item of a user and generates a three-dimensional (3D) virtual fashion item based on the real-world fashion item depicted in the image. The system stores the 3D virtual fashion item in a database that includes a virtual wardrobe comprising a plurality of 3D virtual fashion items associated with the user. The system generates, by the messaging application, an AR experience that allows the user to interact with the virtual wardrobe.

[0008] US2019272675A1 discloses a smart mirror and smart mirror system for mixed or augmented reality display. The system comprises a server and a smart mirror. The smart mirror comprises a display and a camera. The server is configured to receive information associated with a user of the system, identify, using the information, an object for the user, and transmit, to the smart mirror a three-dimensional model of the object.

[0009] EP3465537 discloses a computer implemented method for predicting garment or accessory attributes using deep learning techniques. The method comprises the steps of: (i) receiving and storing one or more digital image datasets including images of garments or accessories; (ii) training a deep model for garment or accessory attribute identification, using the stored one or more digital image datasets, by configuring a deep neural network model to predict (a) multiple-class discrete attributes; (b) binary discrete attributes, and (c) continuous attributes, (iii) receiving one or more digital images of a garment or an accessory, and (iv) extracting attributes of the garment or the accessory from the one or more received digital images using the trained deep model for garment or accessory attribute identification. LIS2015154691 discloses a system and method for virtually fitting an article of clothing on an accurate representation of a user's body obtained by 3D scanning of the user in minimal clothing and in standard garments of known properties. A graphical user interface allows the user to access a database of garments and accessories available for selection for the virtual fitting simulation for which each garment's physical and material properties are known. A finite element analysis is applied to determine the shape of the combined user body and garment and an accurate visual representation of the selected garment or accessory on the proportional model of the user's body based on the analysis is generated.

[0010] Systems including various known aspects are disclosed in CA3116540, WO2017 / 203262, GB2488237 and US12037740.

[0011] CA3116540 discloses systems and methods for generating digital clothing on custom digital avatars. The method includes generating 3-dimensional (3D) representations of one or more garments, wherein the 3D representations of the one or more garments include physical attributes of one or more garment materials, enabling the 3D representations to move and lay on the custom digital avatars in a realistic manner. The method further includes inputting one or more data points for a user. The data points include body measurements of the user, and then the method requires generating, using a processor, a custom digital avatar for the user, wherein the custom digital avatar is configured to approximately conform to the one or more data points’. Using a graphical user interface, one or more garments are selected having an accompanying 3D representation and they are digitally displayed, using a graphical user interface, on the custom digital avatar.

[0012] WO2017 / 203262 discloses a computer implemented method and system for predicting garment or accessory attributes using deep learning techniques. The method comprises receiving and storing one or more digital image datasets including images of garments or accessories. A deep model is trained for garment or accessory attribute identification, using the stored one or more digital image datasets, by configuring a deep neural network model to predict (a) multiple-class discrete attributes; (b) binary discrete attributes, and (c) continuous attributes. GB2488237 discloses a virtual body model of a person created with a small number of measurements and a single photograph and combined with one or more images of garments. The virtual body model is described as representing a realistic representation of the user’s body and is used for visualizing photo-realistic fit visualizations of garments, hairstyles, make-up, and / or other accessories. The virtual garments are created from layers based on photographs of real garment from multiple angles.

[0013] US12037740 discloses a system arranged automatically to generate apparel collection imagery from user-provided imagery. The user-provided imagery includes images of people wearing one or more garments. The system uses segmenting analysis to analyze the user-provided image to identify locations of the garment. From the locations of the garments, the system can determine which garments from an apparel collection can be used to replace those in the user-provided imagery. The system uses pose estimation on the user-provided imagery and modifies a preview image of a replacement garment from the collection. This modified replacement garment image is used to replace the garment in the user-provided imagery.

[0014] According to a first aspect of the present invention, there is provided a method of generating a model of a three-dimensional item, the method comprising: capturing multiple images of the item using a digital device comprising a mobile telephone or a personal digital device; extracting data at the device from the multiple images captured by the device; and using machine learning and the extracted data, generating a three- dimensional base image of the object.

[0015] A method is provided in which a user is able to make use of technology that is commonly available, e.g. a selfie image capture mode, in either still or video mode, to capture plural images of themselves. The technology then further operates by extracting key data points from the captured images and generating an accurate three-dimensional base image based on the extracted key data points. Using machine learning the method then enables the generation of an accurate three-dimensional base image which can be used as the bass for generating composite images including images of clothing. Preferably, though not essentially, LIDAR is used as a means to generate the three-dimensional model. Individual images, which can be still images or frames taken from a video, are used in the generation of the three-dimensional model. As referenced in, for example, Wikipedia, Lidar ("light detection and ranging) is a method for determining ranges by targeting an object or a surface with a laser, and measuring the time for the reflected light to return to the receiver.

[0016] In an embodiment, the multiple images comprise plural images from at least two different angles of the device relative to the item.

[0017] In an embodiment, the three-dimensional base image is a three-dimensional mesh or silhouette.

[0018] In an embodiment, the three-dimensional mesh or silhouette is generated using machine learning.

[0019] In an embodiment, extracting data comprises edge detection and extraction from the captured images. Edge detection preferably includes identifying the outlines or contours of a user’s body from captured images or video frames. The features are used to create a 3D model without storing full visual images of the user, helping preserve privacy.

[0020] In an embodiment, extracted edge data is stored without generating or storing of a complete visual representation of the object.

[0021] The method uses extraction of key data points such as edges and stores these data points, e.g., in database or in a file, which can be used in the generation of the accurate three-dimensional model. Importantly, a complete visual representation of the object, i.e. , the user is not stored which means that the level of data security provided by the system is high.

[0022] In an embodiment, having extracted data at the device from the multiple images captured by the device, the extracted data is used to generate locally at the device the three-dimensional base image of the object. In an embodiment, a generative model is used to learn the appearance of the object based on the extracted data.

[0023] In an embodiment, the generative model is selected from the group consisting of a generative adversarial network (GAN) or a variational autoencoder (VAE).

[0024] A generative model is used, which is trained in the captured data and learns the appearance of the object, i.e. , the user based on the data.

[0025] Learning the appearance or physical parameters of a user means that the system is thus able to analyse user preferences, body shape, past purchases, and interaction data to make intelligent recommendations. This approach ensures that users are presented with options that match their style and fit preferences, streamlining a shopping process.

[0026] In an embodiment, the object is the user of a mobile telephone or a personal digital device.

[0027] According to a second aspect of the present invention, there is provided a method of providing a virtual wardrobe, the method comprising: receiving a three- dimensional model of a user; receiving data relating to an item of clothing; combining the data relating to the clothing with the three-dimensional model of the user to produce an image of the user wearing the identified clothing item, in which the combining of the data relating to the clothing with the three-dimensional model of the user is executed using a physics engine to simulate interaction of the clothing with the shape of the user.

[0028] A method is provided by which a virtual wardrobe can be realised in which a physics engine is used to simulate interaction of the clothing with the shape of the user. A physics engine is used that simulates how selected clothing items will look and fit on a user’s avatar, i.e. the generated three-dimension image of the user or the visual representation of the user, built from the 3D mesh. It can reflect the user’s true proportions and, optionally, aspects of their appearance. It is used to display virtual garments in the try-on experience. The physics engine is arranged and controlled to account for fabric behaviour, drape, and fit under various conditions, offering a true-to-life visualization. Importantly, in some examples the physics engine is used to simulate realistic fabric behaviour including parameters such as drape, stretch, compression and movement.

[0029] Furthermore, the use of a physics engine enables the dynamic simulation of fabric behaviour. This relates in particular to the dynamic interaction of the fabric with the underlying body be that directly the generated silhouette or mesh, or the mesh with a layer of fabric already arranged hereon. The use of a dynamic physics engine is able to determine the way different fabrics might drape, stretch, compress or move, whether in a static pose or in response to motion. A dynamic physics engine is preferably arranged to simulate these effects in real-time or interactively.

[0030] In an embodiment, the data relating to an item of clothing is derived from a third- party database of clothing items.

[0031] In an embodiment, the third-party database is from an online clothing or fashion retailer.

[0032] In an embodiment, the method is executed locally on a user’s mobile digital device.

[0033] In an embodiment, the method comprises storing the produced image of the user wearing the identified clothing item.

[0034] In an embodiment, the three-dimensional model of the user is generated using machine learning based on images captured by the user’s mobile digital device.

[0035] In an embodiment, the three-dimensional model of the user is generated using a method according to the first aspect of the present invention. According to a third aspect of the present invention, there is provided a system for generating a model of a three-dimensional item, the system comprising: an image capture device for capturing multiple images of the item; a processor arranged and configured to extract data at the device from the multiple images captured by the device; and being arranged to use machine learning to generate a three-dimensional base image of the object.

[0036] In an embodiment, the multiple images comprise plural images from at least two different angles of the device relative to the item.

[0037] In an embodiment, the three-dimensional base image is a three-dimensional mesh or silhouette.

[0038] In an embodiment, the three-dimensional mesh or silhouette is generated using machine learning.

[0039] In an embodiment, the system is arranged to execute the method of the first aspect of the present invention.

[0040] According to a fourth aspect of the present invention, there is provided a method of generating a video or media, the method comprising: receiving a three-dimensional model of a user; generating a film including the three-dimensional model of the user, wherein the model is generated by capturing multiple images of the user with a digital device comprising a mobile telephone or a personal digital device; extracting data at the device from the multiple images captured by the device; using machine learning and the extracted data, generating a three-dimensional base image of the user..

[0041] Embodiments of the present invention will now be described in detail with reference to the accompanying drawings, in which:

[0042] Figure 1 is a schematic representation of a flow chart showing the steps in a method of generating a model of a three-dimensional item and a method of providing a virtual wardrobe; Figure 2 shows further steps in a method of generating a model of a three- dimensional item and a method of providing a virtual wardrobe; and

[0043] Figure 3 shows further steps in a method of generating a model of a three- dimensional item and a method of providing a virtual wardrobe.

[0044] The disclosed invention presents a comprehensive system designed to transform the online shopping experience by integrating advanced 3D body scanning technology, virtual try-on capabilities, and intelligent wardrobe management. Utilizing a combination of artificial intelligence (Al), computer vision, augmented reality (AR), and a sophisticated physics engine, this system enables accurate personal avatars and realistic garment simulation. It offers users an unprecedented level of immersion and personalization in online apparel shopping, significantly reducing return rates due to fit issues and enhancing overall customer satisfaction.

[0045] Figure 1 is a schematic representation of a flow chart 2 showing the steps in a method of generating a model of a three-dimensional item and a method of providing a virtual wardrobe. As will be explained the method comprises, initially, generating a model of a three-dimensional item. This can be achieved by a user interacting with a mobile device such as a mobile telephone and capturing multiple images of the item using a digital comprising a mobile telephone or a personal digital device. Data is then extracted at the device from the multiple images captured by the device, and, using machine learning, a three-dimensional base image of the object is generated.

[0046] Referring to Figure at step 4 a user, engaging with their personal digital device, such as a smart phone, opens an App to initiate a scanning process. At step 6, an integrated camera, commonly included of course within smart phones, captures plural images of the user. The App preferably includes instructions to guide a user through stages of image capture and indicative of desired or required angles of capture. Typically, the user is instructed to rotate slowly as images are captured by the mobile phone.

[0047] Next at step 8 data is preferably uploaded to the App which operates to determine and generate a base three-dimensional mesh or silhouette of the user, in dependence on the uploaded data. The three-dimensional mesh or silhouette is the underlying geometric structure of the user’s body, generated from image or depth data. It is preferably used to form the foundation for the digital avatar and can be used for garment simulation and virtual try-on, i.e. process of showing what a garment would look like on the user’s avatar, including how it fits and behaves - it can be a static image or a dynamic simulation, depending on the garment and user interaction.

[0048] An important feature of the present method is that the entirety of the captured data is not uploaded to the App. This ensures a level of data security for the user. The complete images of the user as captured by the mobile device are not uploaded or even accessible to the App.

[0049] Specifically at step 8, the App is arranged first to extract from the captured images essential point or edge data which can be stored locally on the device for use in subsequent processing, as will be described below.

[0050] At the device 10, the stored or generated edge or essential point data is processed to generate the base three-dimensional mesh or silhouette of the user. The processing functionality of the device 10 comprises algorithms that utilise machine learning to generate the base three-dimensional mesh or silhouette. Thus, it can be understood that starting with the local processing on a user’s mobile phone, LIDAR or photogrammetry are used to create the three-dimensional base model. Photogrammetry typically involves capturing multiple 2D images from different angles to reconstruct a 3D model of a user. It allows for scanning using standard cameras without relying on dedicated depth sensors like LIDAR.

[0051] The process of machine learning as executed on the device 10 involves measurements and data appoints being extracted from the stored or generated data. A generative model 14 executed on the user’s device 12 and preferably as part of the provided App is able to learn a user’s appearance based on the extracted data.

[0052] The generative Al model used may be any suitable or appropriate model. Typical examples include a generative adversarial network (GAN) or a variational autoencoder (VAE). These can function as machine learning models that can learn to reconstruct or generate a digital model of the user’s body from image-derived data. Thus, an accurate representation of the user’s physical form can be built by the system.

[0053] Once a user’s appearance has been learned and modelled this generated mesh or silhouette model can then be used to enable virtual fitting of clothes as will now be described with reference to Figure 2.

[0054] The process of virtual fitting uses a clothing simulation 18 of a retailer’s catalogue. As is well known, the retailer’s online catalogue will typically include graphical representations of clothing items that the retailer wishes to sell, and these can be perused and viewed by users in known ways. When a user makes a selection of an item of clothing from the catalogue, a physics and rendering engine 20 is operated to simulate how the different materials and clothes would fit or drape over a user based on their generated three-dimensional model (“avatar”). Once an item of clothing is selected and by operation of the physics engine and rendering engine a simulation is created. This is output 26 for the user to see on the screen or interface of the mobile phone. The user is then able to confirm their choice and make an online purchase or decline if they do not want to make the purchase.

[0055] The physics and rendering engine(s) 20 are arranged to provide any or all of texturing, rigging, skinning, animation and lighting to generate a representation of what the user would look like wearing the item of clothing. The rendering engine in examples is arranged to generate the final visual output turning the mesh, clothing and physics data into a realistic on-screen image or animation.

[0056] The physics engine is preferably programmed with data relating to the different materials and their weights and other relevant parameters which will affect how they appear and interact with user’s body when worn. Thus, the physics engine preferably serves to simulate realistic fabric behaviour including parameters such as drape, stretch and compression. Preferably, the physics engine is arranged and configured to enable the dynamic simulation of fabric behaviour. This relates in particular to the dynamic interaction of the fabric with the underlying body, be that directly the generated silhouette or mesh, or the mesh with a layer of fabric already arranged hereon. The incorporation of a dynamic physics engine is able to provide a user with a simulation of how the garment will look, not only in a static arrangement but also as it folds or moves with normal wear.

[0057] Thus, an accurate representation of the user’s avatar “wearing” the item can be created.

[0058] At 24 purchased items, or even liked items, can be stored in a virtual wardrobe. This can be data store on a user’s mobile phone which stores records and menus of all clothing items. It is preferably further arranged to display the clothing items appropriately, e.g. as they would look when worn. It provides a user with flexibility and choice in terms of deciding what to wear at any point in time and also in creating outfits based on combinations of clothing items.

[0059] Thus, the virtual wardrobe uses the initial steps of three dimension model generation, and then a physics and rendering engine to generate accurate representations of a user virtually wearing the clothes. This is a significant improvement on systems that, say simply show images of the clothing items, or attempts to portray the clothe sin three dimensions. The user is able to make decisions based on a data that shows them what they would actually look like when wearing the clothes in question, in a way not previously possible.

[0060] The operation of the virtual wardrobe can be understood clearly with reference to Figure 3. As seen steps 12 and 14 are the same as those described with reference to Figure 1. Step 14, which constitutes the execution of the generative Al model such as a GAN or VAE operates gradually to improve its knowledge and understanding of the user. The details of the three-dimensional model are improved 28 as the Al generative system such as the GAN or VAE is exposed to more user data over time.

[0061] The data stored locally on a user’s device can be shared 30 with a retailer when desired by a user. This enables the retailer to make suggestions or recommendations based on the data to provide a user with options of clothing items to purchase. This interaction can be controlled by a user and only operates when the user so desires. An example of the virtual wardrobe interface 32 is shown schematically as being dependent on the three-dimensional model of the user and the selections or suggestions of clothing by a retailer. The three-dimensional virtual wardrobe functions by overlaying the locally-stored (i.e. on the user’s mobile device) three-dimensional model of the user onto the wardrobe. This enables a user to interact with the virtual wardrobe interface and see renders of what they look like wearing particular clothing items. As explained above, the hang of the clothing on the avatar will be faithful and realistic based on the calculations performed by the physics and / or rendering engines.

[0062] In practice then a user is provided with a virtual wardrobe that can, in real time, generate realistic and technically faithful, i.e. correct, representations of what they will look like wearing a particular item of clothing. This is achieved without the sharing of personal data (regarding the appearance of the user) with a retailer. This can be particularly helpful as often it is not until a user actually tries on a piece of clothing that they get a true idea of whether they like it. This process of seeing what they actually look like in an article of clothing is achieved virtually using the present system and method.

[0063] As described above, a generative model 14 executed on the user’s device 12 and preferably as part of the provided App is able to learn a user’s appearance based on the extracted data and generate the 3-dimensional model of the user. As also described above one application to which the generated model can be applied is that of a virtual wardrobe. Another use is in the generation of personalized advertising content, as will now be described.

[0064] The generative model 14 is executed on a user’s device which, it will be understood, means that the processing is done locally to the user. It could be a device under control of the user but not the original device that, say, captures the images. For example, a typical set up might be that a user has a mobile telephone that they use to capture the images and then they connect the mobile telephone to their local personal computer to perform, locally, the described processing. In other words, it could be a personal computer to which a user connects the image capture device. The data is not sent to a remote server, e.g., on a network, or to any device that is not under the user’s direct control. Preferably the personal digital device is the user’s actual mobile telephone.

[0065] In another example, it is envisaged that the captured images could be sent to a remote server and the extraction of data is performed at the remote server. However, it is preferred that it is done locally.

[0066] In this example, the high-fidelity digital avatars of users, generated through the above-described advanced 3D body scanning and rendering technology, is used to personalize advertising content. A user’s generated model, i.e. , avatar, is substituted into an existing advertisement. This is achieved by the application of an Al video generator which is able either to modify existing video advertisements by replacing characters in the original film with the user’s avatar, or by generating entirely new video content including the user’s avatar and whatever product or items are being advertised.

[0067] In another example, the generative model can be arranged to create a personalised vignette based on the high-fidelity digital avatars of a user.

[0068] Thus, a uniquely tailored marketing experience is created which is able to resonates with an individual, enhancing engagement and conversion rates.

[0069] To achieve this, the detailed generated 3D model created by the abovedescribed system, is integrated into various advertising mediums such as video ads, interactive web banners, and virtual reality (VR) or augmented reality (AR) experiences.

[0070] Advanced image processing and AR / VR integration techniques are used to ensure seamless insertion of a generated avatar into marketing materials, preserving the original lighting, perspective, and context for a realistic appearance.

[0071] Through use, machine learning algorithms are arranged to analyse user preferences and behaviour to select dynamically and personalise advertisements in realtime, ensuring relevance and increasing the efficacy of marketing efforts. Applications of this enable a user e.g., a customer, to “see” themselves in the clothes, using the gadgets, or experiencing the services being advertised, offering an immersive preview of the product’s impact on their lives.

[0072] For brands, this presents an opportunity to form a stronger emotional connection with their audience, leading to enhanced brand loyalty and consumer satisfaction.

[0073] In a further application, the generated model is used in social media platforms, e- commerce sites, and digital content providers. By the machine learning based model generation described herein, a broad range of technical uses are enabled. In addition, application of personalised marketing is enabled from personalised product recommendations to customised advertising narratives that feature the consumer as the protagonist.

[0074] The system in preferred embodiments, can thus be understood to include any or all of the following aspects:

[0075] Innovative 3D Body Scanning

[0076] Leveraging the latest in photogrammetry and depth sensing technology, including LiDAR sensors available on select smartphones and devices, the system captures precise body measurements. It constructs highly accurate 3D models of users, forming the foundation for personalized virtual try-on experiences.

[0077] Virtual Try-On with Realistic Simulation

[0078] Employing AR and VR technologies alongside a cutting-edge physics engine, the system simulates how selected clothing items will look and fit on the user’s avatar. This simulation accounts for fabric behaviour, drape, and fit under various conditions, offering a true-to-life visualization. Intelligent Wardrobe Management

[0079] In addition to virtual try-on, the system incorporates a wardrobe management component. It catalogues users’ existing apparel into a virtual wardrobe, suggests outfits based on personal style, occasion, and weather conditions, and enables virtual try-on of owned items for complete outfit planning.

[0080] Al-Driven Personalization and Recommendation

[0081] The core Al algorithms analyse user preferences, body shape, past purchases, and interaction data to tailor clothing recommendations. This Al-driven approach ensures that users are presented with options that match their style and fit preferences, streamlining the shopping process.

[0082] Cross-Platform Compatibility

[0083] Designed to function seamlessly across various devices, including smartphones, AR / VR headsets, and smart mirrors equipped with cameras, the system ensures accessibility and convenience for a broad user base.

[0084] Advanced Technologies and Implementation

[0085] The system’s architecture integrates several key technologies: CNNs for image processing, GANs for texture and pattern synthesis, geometric deep learning for 3D structure understanding from 2D images, and reinforcement learning for dynamic adjustment based on user feedback. Conclusion: This invention addresses critical challenges in the online apparel industry by offering a sophisticated solution that marries accuracy in body modelling with the immersive experience of AR / VR. It paves the way for a future where online shopping is not only convenient and efficient but also highly personalized and engaging.

[0086] The present inventive system incorporates a temporal body composition analysis module that processes and analyses user physique changes over time. In particular, it refers in examples and embodiments to the analysis of how a user’s body changes over time, based on a sequence of 3D scans and / or biometric data. The system can identify trends such as muscle growth, fat loss or postural changes by comparing the data across different points in time. The module is built into the software that operates the method more broadly as described herein. It could also be provided as a bespoke hardwired piece of logic.

[0087] The description from hereon will describe the temporal body composition analysis module as a software module but it will be understood that it can also be provided in hardware. The software module comprises each of:

[0088] 1. A time-series data processing engine;

[0089] 2. A biometric data integration system; and

[0090] 3. A predictive modelling framework

[0091] The function of each of these will now be described in detail.

[0092] Time-series Data Processing Engine

[0093] The time series data processing engine is a software engine that serves to track changes in user physique over time using time-series data analysis. Furthermore preferably it is arranged to integrate biometric data from wearable devices into the analysis of user physique changes. For example, the system could include heart rate variability and motion tracking. The engine then serves to enable predictive modelling including body composition changes, which can include muscle development or atrophy and biomechanical stress analysis.

[0094] As a first step the engine captures and stores sequential body measurements. As explained above in typical applications of the broader method a user engages with their personal digital device, such as a smart phone, and opens an App to initiate a scanning process. The integrated camera of the device captures plural images of the user and the App uses these in the generation of the silhouette or three-dimensional mesh of the user.

[0095] In generating the mesh or the silhouette the App, through use of the time-series data processing engine is able and arranged to track volumetric changes in specific body regions. For example, it could track whether a user’s waist has changed or not. Further, it is used to determine changes in muscle strength, size, orientation and the like.

[0096] By doing this, the module is arranged to track muscle development and atrophy patterns.

[0097] A longitudinal biometric data stream system can be used. This refers to the ongoing or periodic collection of biometric data from wearable devices, such as smartwatches or fitness trackers. This includes information like heart rate variability, resting heart rate, sleep patterns, activity levels, and step count. These data points are time-stamped and linked to the user’s 3D scan history, enabling the system to analyse physical changes over time. This, in turn, supports predictive modelling for body composition, posture and muscle development, and helps inform avatar updates, garment recommendations and fit forecasting.

[0098] Biometric Data Integration System

[0099] A preferred feature of the present system is the ability and arrangement to integrate biometric data to the system. This is achieved with the use of interface with a user’s wearable devices such as a smart watch or the like which function, amongst other things, to detect biometric data from the user. Secured APIs are provided as part of the system which are configured, preferably continually, to download or transfer data from a connected user device to the App in the mobile telephone. The biometric data integration can involve pulling in or transferring physiological data (e.g. heart rate, movement data and sleep patterns) from external devices like smartwatches and combining it with scan history to build a more complete picture of a user’s physical state.

[0100] As well as biometric data the APIs are preferably arranged to securely access data from the user’s wearable device data such as acceleration and impact patterns. The gathered biometric and other user data is analysed and processed to enable correlation of the physiological measurements with any body composition changes detected by the time series data processing engine. Predictive Modelling Framework

[0101] In addition there is provided a predictive modelling framework which advantageously serves to predict body chape and composition changes based on gathered data from the time series data processing engine in combination with the biometric data.

[0102] This can be done based on Al models such as learnt responses to large gathered data sets. Typically, though the module is arranged to function to generates forwardlooking body composition projections. This can be helpful if the system as a whole is being used to purchase or model clothes since sizing can be selected appropriately taking into account the predicted model framework.

[0103] The predictive model framework further serves to a analyse repetitive motion impacts, evaluate bio-mechanical stress patterns and thereby enable prediction of potential physiological adaptations.

[0104] Embodiments of the present invention have been described with particular reference to the examples illustrated. However, it will be appreciated that variations and modifications may be made to the examples described within the scope of the present invention.

Claims

Claims1. A method of generating a model of a three-dimensional item, the method comprising: capturing multiple images of the item using a digital device comprising a mobile telephone or a personal digital device; extracting data at the device from the multiple images captured by the device; using machine learning and the extracted data, generating a three-dimensional base image of the object.

2. A method according to claim 1, comprising, using a temporal body composition analysis module to track changes in user physique over time in dependence on the generated three-dimensional base image of the object.

3. A method according to claim 2, comprising, using the temporal body composition analysis module, to map muscle development and atrophy patterns in dependence on the tracked changes in user physique.

4. A method according to claims 2 and 3, comprising receiving biometric data relating to the user from connected devices such as a smart watch or heart sensor.

5. A method according to any of claims 2 to 4, comprising generating predictive models for body changes in dependence on the tracked changes in user physique.

6. A method according to any of claims 1 to 5, in which the multiple images comprise plural images from at least two different angles of the device relative to the item.

7. A method according to any of claims 1 6, in which the three-dimensional base image is a three-dimensional mesh or silhouette.

8. A method according to claim 7, in which the three-dimensional mesh or silhouette is generated using machine learning.

9. A method according to any of claims 1 to 8, in which extracting data comprises edge detection and extraction from the captured images.

10. A method according to claim 9, in which extracted edge data is stored without generating or storing of a complete visual representation of the object.

11. A method according to any of claims 1 to 10, in which, having extracted data at the device from the multiple images captured by the device, the extracted data is used to generate locally ay the device the three-dimensional base image of the object.

12. A method according to claim 11, in which a generative model is used to learn the appearance of the object based on the extracted data.

13. A method according to claim 12, in which the generative model is selected from the group consisting of a generative adversarial network (GAN) or a variational autoencoder (VAE).

14. A method according to any of claims 1 to 13, in which the object is the user of a mobile telephone or a personal digital device.

15. A method of providing a virtual wardrobe, the method comprising: receiving a three-dimensional model of a user; receiving data relating to an item of clothing; combining the data relating to the clothing with the three-dimensional model of the user to produce an image of the user wearing the identified clothing item, in which the combining of the data relating to the clothing with the three-dimensional model of the user is executed using a physics engine to simulate interaction of the clothing with the shape of the user.

16. A method according to claim 15, in which the data relating to an item of clothing is derived from a third-party database of clothing items.

17. A method according to claim 16, in which the third-party database is from an online clothing or fashion retailer.

18. A method according to any of claims 15 to 17, in which the method is executed locally on a user’s mobile digital device.

19. A method according to claim 18, comprising storing the produced image of the user wearing the identified clothing item.

20. A method according to any of claims 15 to 19, in which the three-dimensional model of the user is generated using machine learning based on images captured by the user’s mobile digital device.

21. A method according to claim 20, in which the three-dimensional model of the user is generated using a method according to any of claims 1 to 14.

22. A method according to any of claims 15 to 21, in which the physics engine is arranged and controlled to simulate fabric behaviour including one or more of drape, stretch and compression.

23. A method according to claim 22, in which the physics engine is controlled to enable the dynamic simulation of fabric behaviour, such as dynamic interaction of the fabric with the underlying model.

24. A method according to claim 23, in which the dynamic physics engine is arranged to simulate of how the garment will look, in a static arrangement and in addition as it folds or moves in use.

25. A system for generating a model of a three-dimensional item, the system comprising: an image capture device for capturing multiple images of the item; a processor arranged and configured to extract data at the device from the multiple images captured by the device; and being arranged to use machine learning to generate a three-dimensional base image of the object.

26. A system according to claim 25, in which the multiple images comprise plural images from at least two different angles of the device relative to the item.

27. A system according to claims 25 or 26, in which the three-dimensional base image is a three-dimensional mesh or silhouette.

28. A system according to claim 27, in which the three-dimensional mesh or silhouette is generated using machine learning.

29. A system according to any of claims 25 to 28, arranged to execute the method of any of claims 1 to 14.

30. A method of generating a video or media, the method comprising: receiving a three-dimensional model of a user; generating a film including the three-dimensional model of the user, wherein the model is generated by capturing multiple images of the user with a digital device comprising a mobile telephone or a personal digital device; extracting data at the device from the multiple images captured by the device; using machine learning and the extracted data, generating a three-dimensional base image of the user.

Images