System, platform and method for personalized shopping using automated shopping assistants

Abstract

To provide a personalized shipping assistant system, a platform, and a method.SOLUTION: A personalized shopping assistant system includes: a point of sale-based automated shopping assistant apparatus for generating a 3D user shopping profile on the basis of a capture of at least a part of a user's body; a shopping assistant server connected to a shopping assistant database based in a communications cloud, wherein the user shopping profile is uploaded to the shopping assistant database; a mobile device application for applying the user shopping profile in a store by using the user's mobile device; and scannable product data and a plurality of product identifiers stored on the shopping assistance database for capturing selected products.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) This application claims priority from U.S. Provisional Patent Application No. 62,443,275, entitled "A SYSTEM, PLATFORM AND METHOD FOR PERSONALIZED SHOPPING USING AN AUTOMATED SHOPPING ASSISTANT," filed January 6, 2017, which is incorporated by reference in its entirety.

[0002] The present invention relates generally to methods, applications and apparatus useful in personalized product shopping. [Background technology]

[0003] The vast majority of shoe shopping today still takes place in traditional stores. Most customers are familiar with the limitations of traditional shopping, such as the store manager and sales clerk of the day. Typically, customers rely on sales clerks to guide them to products and help them locate products in stock, try on products, etc.

[0004] Moreover, the typical shopping experience essentially requires a customer to repeat each visit to the same or different stores, resulting in significant inefficiency and user frustration.

[0005] It would be highly advantageous to have a system or method that could enable a highly accurate and user friendly automated or semi-automated fitting solution, both online and in-store. Summary of the Invention [Means for solving the problem]

[0006] According to one embodiment of the present invention, there is provided an apparatus, system, and method for providing personalized online product fitting according to some embodiments.

[0007] The method for personalized shopping includes the steps of an automated shopping assistant system accessing product data, a matchmaking system accessing user history data, the matchmaking system accessing user preference data, and the matchmaking system accessing user anatomical data obtained from the automated shopping assistant device, such that the automated shopping assistant system matches the user history, preference and anatomical data with the product data to generate a personalized matching system.

[0008] In some embodiments, the automated shopping assistant system matches user history and preference data with product data to generate personalized product recommendations.

[0009] In some embodiments, an automated shopping assistant system may be used to enable users to order personalized products.

[0010] In some embodiments, the automated shopping assistant system may be used to provide a simulation representative of one or more anatomical features of a user.

[0011] In one embodiment, the user and one or more third parties may provide product fitting feedback, and the user and one or more third parties may provide social feedback. The automated shopping assistant system may tailor personalized products based on the product feedback. The system may also provide anatomical data about the user, and the automated shopping assistant system may also take the anatomical data into account to generate a user shopping avatar that includes one or more characteristics of the user.

[0012] The automated shopping assistant system may include a virtual try-on feature. In one embodiment, before the user orders a personalized product, the product is generated based on the avatar. The user preferences may be selected from the group consisting of size, color, material, and type preferences.

[0013] According to some embodiments, a platform for personalized shopping is provided, comprising: a cloud-based server including a profile module for generating digital avatars for a plurality of users, a product module for integrating product data for a plurality of products, and a match-making module adapted to execute code for matching the digital avatar data with the product data to generate product recommendations; an end-user computing device communicatively connected to the cloud-based server and including an image capture element, the match-making module executing a software application to generate a user mobile shopping avatar based on capture of at least a portion of the user's anatomical structure for use in generating anatomical data for a digital avatar profile for the user.

[0014] In some embodiments, the platform is adapted to generate and / or present a simulation representative of one or more anatomical features of the user.

[0015] In some embodiments, the platform may have a product ordering module, a product customization module, a social shopping module, and / or a product fitting module, etc.

[0016] A handheld system for personalized shopping may have a screen configured to receive user input, a device camera for capturing standard images of the user's anatomy, and a processor having a register adapted to analyze anatomical data, product data, user history data, and user preference data, the processor configured to obtain information from the register, write information to the register, and match the user history and preference data with the product data to generate a personalized matching system, the processor configured to match the user history and preference data with the product data to generate a personalized product, and a user may purchase the personalized product by providing user input to the screen.

[0017] In one embodiment, the system has a depth sensor configured to accurately determine the distance of an object from the sensor, and the depth sensor information is used to match the preference data with the product data. In some embodiments, the depth sensor enables providing a 3D scan of at least a portion of the body associated with the user's anatomical profile to capture the length, width and depth of that portion of the body.

[0018] In some embodiments, the handheld system includes a software application executing on the handheld system for generating and presenting a graphical simulation of a user mobile shopping profile based on a capture of at least a portion of the user's anatomical structure.

[0019] A shopping assistant system for shopping using a shopping profile is provided, including a POS-based automated shopping assistant device for generating a user shopping profile; a mobile device app for applying the user shopping profile in-store; a communication cloud-based shopping assistant server connected to a shopping assistant database; and a plurality of product recognition devices, such as tags, for enabling a user to capture selected products.

[0020] The shopping assistant device may include one or more image scanners to capture at least a portion of the body associated with the user's anatomical profile to enable capturing the length, width and depth of that portion of the body.

[0021] The shopping assistant may include a product module for aggregating product data for multiple products, and a matchmaking module adapted to execute code that connects the shopping profile and the product data to generate product recommendations.

[0022] The shopping assistant may include order tags associated with the products for a user to order by capturing the order tag.

[0023] The shopping assistant may include a proximity sensor associated with the shopping assistant device to identify the user when the user enters a selected geographic area around the shopping assistant device.

[0024] The shopping assistant may include a proximity sensor associated with the shopping assistant device to enable automatic triggering of recognition and scanning when a user is appropriately positioned near the proximity sensor.

[0025] The shopping assistant may include a virtual try-on module to digitally try on selected products in a user shopping profile.

[0026] A method for in-store shopping enhancement is provided, comprising: identifying a user entering a shopping area using a proximity sensor associated with an automated shopping assistant device; connecting the user to an automated shopping assistant system to open a user profile; initiating capture of one or more body parts by the automated shopping assistant device using one or more image sensors; generating a user 3D shopping profile for the user; initiating downloading of an automated shopping assistant application onto the user's mobile computing device; connecting the automated shopping assistant device to a user mobile device to close the user shopping profile user mobile device; selecting a product of interest by the user by capturing a products tab using the user mobile device; and providing product related information for the selected product to the user.

[0027] The method may further include ordering the selected product using the user mobile device.

[0028] The method may further include customizing the selected product using the user mobile device.

[0029] The method may further include ordering the customized product using the user mobile device.

[0030] The method may further include providing product inventory information to a user.

[0031] The method may further include generating a user profile avatar.The method may further include having the user shopping profile avatar try on the selected product.

[0032] The method may further include transmitting the tried-on user profile avatar to a social media system for viewing by selected peers with whom the user is connected. The method may further include enhancing the user shopping profile based on the user behavior.

[0033] The method may further include providing shopping recommendations based on the user shopping profile and / or user shopping behavior.

[0034] The method may further include using the user shopping profile for shopping at the online store.

[0035] The method may further include enhancing a user shopping profile with additional imaging devices.

[0036] The method may further include generating an additional user shopping profile using the mobile device.

[0037] The method may further include transferring the user profile data, with user consent, from the in-store device to a user device, such as a smartphone application.

[0038] Provided herein is a method for personalizing shopping, comprising: an automated shopping assistant system accessing product data; an automated shopping assistant system accessing user history data; an automated shopping assistant system accessing user preference data; an automated shopping assistant system accessing anatomical data regarding a user, and a matchmaking system executing a software application to generate a user mobile shopping avatar based on a capture of at least a portion of the user's anatomical structure; and The Ping Assistant system matches the user history data, the preference data, the shopping avatar and the product data to generate a personalized matching system.

[0039] The method may further include generating personalized product recommendations.

[0040] The method may further include ordering the personalized product.

[0041] The method may further include the step of the user and one or more third parties providing product feedback.

[0042] The method may further include the step of the user and one or more third parties providing social feedback.

[0043] The method may further include the automated shopping assistant system adjusting the personalized product based on the product feedback.

[0044] The method may further include providing a simulation representative of one or more anatomical features of the user.

[0045] The method may further include a virtual try-on feature.

[0046] The method may further include ordering a personalized product, the product being generated based on the user shopping avatar.

[0047] The method may further include selecting a product type from the group consisting of size, color, material, and type preferences.

[0048] The principles and operation of systems, apparatus and methods according to the present invention may be better understood with reference to the drawings and the following descriptions, it being understood that these drawings are provided for illustrative purposes only and are not intended to be limiting. [Brief description of the drawings]

[0049] [Figure 1A] FIG. 1 is a schematic system diagram illustrating a system for facilitating personalized shopping, according to some embodiments. [Figure 1B] FIG. 1 is a schematic system diagram illustrating a platform for facilitating personalized shopping, according to some embodiments. [Diagram 2] 1 is a flow diagram illustrating a process for facilitating personalized shopping according to some embodiments. [Diagram 3] 1 is a flow diagram for facilitating a personalized in-store shopping experience according to some embodiments. [Figure 4] 1 is a flow diagram for facilitating a personalized online shopping experience according to some embodiments. [Diagram 5] 1 is a flow diagram illustrating an example of a personalized hybrid shopping experience according to some embodiments. [Figure 6] 1 is a flow diagram illustrating an example of personalized shoe shopping, online or offline, according to some embodiments. [Figure 7] 1 is a flow diagram illustrating an example of personalized eyewear shopping and manufacturing, online or offline, according to some embodiments. [Figure 8A] 1 illustrates an exploded view of an example shopping assistant device, according to some embodiments. [Figure 8B] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8C] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8D] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8E]1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8F] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8G] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 8H] 1 illustrates a set of diagrams of an example shopping assistant device, according to some embodiments. [Figure 9] 1 illustrates a shopping assistant system according to some embodiments. [Figure 10] 1 is a flow diagram illustrating an example of integrated use of a shopping assistant in a store device and a remote user communication device with a personalized shopping support application, according to some embodiments. [Figure 11A] According to some embodiments, an example process for using a shopping assistant is described. [Figure 11B] According to some embodiments, an example process for using a shopping assistant is described. [Figure 12A] 13 is an example screenshot showing an interactive screen on a shopping assistant screen to guide a user to place their feet on a marked pad, according to some embodiments. [Figure 12B] 13 is an example screenshot showing an interactive screen on a shopping assistant screen to guide a user to place their feet on a marked pad, according to some embodiments. [Figure 13A] 13 is an example screenshot showing an interactive guide on a shopping assistant screen or mobile screen to assist a user in defining their profile and contact information according to some embodiments. [Figure 13B]13 is an example screenshot showing an interactive guide on a shopping assistant screen or mobile screen to assist a user in defining their profile and contact information according to some embodiments. [Figure 14A] 13 is an example of a screenshot on a shopping assistant screen or mobile screen to show a simulated rendition of a scanned pair of feet and lower legs, according to some embodiments. [Figure 14B] 13 is an example of a screenshot on a shopping assistant screen or mobile screen to show a simulated rendition of a scanned pair of feet and lower legs, according to some embodiments. [Figure 15A] FIG. 13 is an example of a screenshot on a shopping assistant screen or mobile screen to assist a user in entering behavior-related information that can be used to provide better user-relevant output. [Figure 15B] FIG. 13 is an example of a screenshot on a shopping assistant screen or mobile screen to assist a user in entering behavior-related information that can be used to provide better user-relevant output. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0050] The following description is presented to enable one skilled in the art to make and use the invention as provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Thus, the present invention is not intended to be limited to the specific embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the invention.

[0051] The term "fitting" as used herein refers to trying on a product, seeing a product try on, or modifying a product to a particular person's body or other physical parameters. The term "avatar" refers to a representation, personification, icon, model, or statue of a particular person, especially for representing a person on a screen.

[0052] The relatively low rate of online shopping footwear purchases can be increased by providing customers with accurate size recommendations for each shoe model, optionally based on a simulation or avatar model of the customer's foot, resulting in greater customer confidence in purchasing.

[0053] Non-limiting embodiments of the present invention include systems, platforms, and methods for facilitating highly personalized shopping, including effective try-on of products, whether online and / or in-store. In some embodiments, systems, platforms, and methods are provided for enabling personalized manufacturing of products.

[0054] Reference is now made to FIG. 1A, which is a schematic system diagram illustrating a system 100 for facilitating personalized shopping, according to some embodiments. The system 100 enables seamless in-store and / or online shopping using highly accurate user shopping profiles and / or avatars generated by the automated shopping assistant device. In some cases, data processing is performed on the cloud and / or locally on the automated shopping assistant device.

[0055] As shown in the figure, the personalized shopping system 100 includes a platform for personalized shopping profile management 105, which may include a digital avatar profile module 110, a digital product file module 115, a product selection module 120, a product fitting module 125, a social shopping module 130, a product ordering module 135 and a product customization module 137.

[0056] The platform 105 is in communication with a communications cloud 140, which may include a physical profile data module 145 communicatively connected to a launchpad, kiosk, or automated shopping assistant device 185, which is further communicatively connected to a user 180 to provide physical user data, for example, from 2D and / or 3D scans or other digital measurement sources. The communications cloud 140 may further include a user preference data module 150 communicatively connected to the automated shopping assistant device 185 and / or 180 to provide user preference data. The communications cloud 140 may further include a product file data module 160 communicatively connected to a product database 165, and a product matching data module 170 communicatively connected to a product database 175, which includes a product matching algorithm.

[0057] In one embodiment, device 185 includes one or more imaging devices, such as 2D and / or 3D cameras, which may be movable or fixed components. Device 185 may further include one or more sensors, such as a proximity sensor, a scanner, a camera, a pressure plate, and / or other sensors.

[0058] As shown, the digital avatar profile 110 is highly personalized and constructed from various data sources, whether directly or indirectly from the user, and whether representing physical characteristics and / or mental, emotional, psychological characteristics. The digital avatar profile 120 generally includes a file or group of files and data points on which instructions can be executed to enable the generation of a high-resolution user profile or avatar from one or more data sources. Additionally, the product selection module 120 generally matches the personalized avatar profile 110 with selected digital products, whether online or offline. Calculations for the matching algorithm may be performed by a processor through the storage, retrieval, and processing of data. In some embodiments, one or more device registers may be used, where a register(s) refers to one of a small set of data-holding locations that are part of a computer processor for holding computer instructions, storage addresses, or any type of data. The matching algorithm may include executing code to provide a perfect or near perfect match between product types, sizes, styles, etc., provided by a product database and physical parameters and / or user preference data as defined by a user avatar profile based on a scan of at least a portion of the user's body associated with the user's profile. For example, User A may have a foot profile defined by size, width, and depth. The user's profile may further include preference data for preferred style, shoe type, and color, e.g., blue or gray athletic shoes. The product database may include, for example, athletic shoes in colors blue, gray, and blue-gray that fit or nearly fit the user's size, width, and depth. In this case, a matching algorithm matches the profile definition with one or more products from the product database that match the user profile.

[0059] The product selection module 120 may be further improved by providing system feedback and product fitting data using the product fitting module 125, as well as social shopping data from the social shopping module 130. The product fitting module, in some embodiments, includes a means for feedback from a shopping assistant or supporter with the shopper. In other embodiments, the product fitting module includes a means for feedback from a virtual shopping assistant or supporter, e.g., a communicatively connected supporter, a digital or virtual mirror or screen that displays the user with the product. Additionally, the product customization module 137 may receive data from the product fitting module 125 and / or the social shopping module 130 to help further personalize the product being considered for acquisition according to the digital avatar and the product fitting module 125 and / or the social shopping module 130. The product customization module 137 may enable the user to modify or customize the product being tried on or sampled, for example, by changing the color, shape, design, size, material, etc. of the product. In this manner, the product being ordered may be built according to the user's specific or customized requirements. Additionally, the product customization module 137 may send the customized product as selected or generated by the user to the product selection module 120, which may then initiate an order for the customized product via the product order module 135. Moreover, user updates embodied in the user's modifications made in the product selection module 120 may be used to update the digital avatar profile 110, thereby keeping the user avatar profile up to date with, for example, the user's physical changes, changing preferences, etc.

[0060] The product selection module 120 includes files or groups of files and data points that can execute instructions to execute commands to enable matching of high-resolution user profiles or avatars with products that fit the shopping research being performed by each system user at a high level. This module can further integrate feedback generated in the system modules to continually improve the accurate product recommendations it provides. Using various technical procedures performed on the avatars, such as integrating length, width, height and additional distances, as well as volume, cross-sectional area and circumference, the system can represent an array of numbers of avatars. This array of numbers can, in some embodiments, represent various elements of the avatar to enable comparison with similar elements of products tried on by the avatar. When the avatar data is compared to product data from the product file, which can be enhanced using recommendation algorithms and machine learning techniques, etc. accordingly, this can enable accurate, continually improving, personalized product recommendations from the system.

[0061] According to some embodiments, the digital avatar profile can be generated with a relatively low level of integration of a 3D scanner. In some examples, sensors or scanners that can be used can include structured illumination, time-of-light, photogrammetry, or any other type of 3D and / or 2D scanning technology. Suppliers of such technology include, but are not limited to, PrimeSense™ based scanners, Occipital Structure Sensor, 3D-Systems Sense and iSense™ sensors, Intel™ RealSense sensors (standalone or integrated into the machine), iPad® or tablet based Scanning Platform, PC (integrated and external), Android+RealSense (next generation) devices, and Google Project Tango devices.

[0062] Reference is now made to FIG. 1B, which is a schematic system diagram illustrating a platform 180 for facilitating personalized shopping, according to some embodiments. As shown in the figure, the personalized shopping platform 180 includes one or more user mobile devices 182, such as, without limitation, a smart phone or tablet, including a camera, an application, and a data connection; a store or point of sale device, vending machine, or automated shopping assistant device 184, typically an electronic device at or near a store, such as, for example, with one or more cameras, sensors or scanning devices, applications, and a data connection; and the mobile device(s) 182 and / or the automated shopping assistant device 185 connect to a communications cloud 186. The communications cloud 186 includes a digital product module 190 for holding, consolidating, and otherwise managing data for multiple products; a digital avatar profile module 188 for holding, consolidating, processing, and otherwise managing profile data for multiple users; and a matching module 192 for matching product and avatar profile data to assist in enabling product recommendations and other matching functions. The platform 180 further includes a product customization module 194 for enabling ordering of customized products based on the matching module output and / or user selections; and a product ordering module 196 for enabling ordering of products based on the matching module output and / or user selections.

[0063] Reference is now made to FIG. 2, which is a flow diagram illustrating a process for facilitating personalized shopping, whether online or offline, according to some embodiments. As shown in the figure, product information from a product database 200 may be used to find, purchase, or manufacture a product. At step 205, historical data for the user may be obtained, for example, based on previous user purchases and research. At step 210, user preference data, such as size, color, material, type preferences, etc. may be obtained. At step 215, scanned or graphic data may be obtained for the user, for example, from standard photographs, 2D and / or 3D image scanning, or from capture and processing using a POS automated vending machine or automated shopping assistant device, etc. This graphic data is used by the personalized shopping system to generate a user physical profile based on the user's physical characteristics. At step 220, a multi-dimensional user shopping profile (hereinafter referred to as a user shopping avatar) may be developed by processing the various input data from steps 205, 210, and 215, thereby generating a user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. The profile or avatar is a dynamic structure that may be continually improved in a manner that animates, reflects or represents the user, optionally using feedback and additional input from steps 205, 210, and / or 215. In one embodiment, the user avatar may be used to match the user with potential products within any retail store in a chain, or online, within a single store, and / or within multiple stores, for example, in a network or partnership of online and / or offline stores or platforms.

[0064] In step 225, matchmaking of the user shopping profile to the products being researched or requested is performed. In this step, product data from the product database 200 is matched to the products being researched by the user according to the particular user shopping profile, thereby enabling a high degree of filtering out products that are not suitable for the particular user and a high degree of matching suitable products according to the particular user's personal shopping profile and preferences. The matching step may be complemented by providing recommendations to the user based on the aforementioned user profile to product matchmaking process. In some embodiments, generation of matches between the generated user profile and multiple products that fit one or more elements of the generated profile is provided, thereby enabling, for example, fit and size recommendations and additional data connecting the profile and the product(s).

[0065] At step 230, product fitting data from physically present personnel or remote people's feedback may be used to help correct the matchmaking of user data to product data, e.g., feedback from in-store salespeople or remote people connected, e.g., via a smartphone or computer, may be used to update the user profile. In some cases, feedback from salespeople or friends, e.g., which colors look good or which sizes look best, may be used by the user to update their shopping profile. In some cases, advanced graphics processing and 3D rendering may be used to allow the user to virtually try on the products being researched, so that the user can see themselves wearing the products according to a digital simulation of placing the products on the user's shopping avatar. In some cases, the system may provide static or dynamic high-resolution visual output, such as animated avatars or characters, rendered images and / or virtual representations of recommendations, optionally tried on by the avatar. For example, such depictions may allow the user to see the product being tried on depicted on the avatar, thereby assisting the user in visualizing details such as fit, tightness, color, style, material, etc., according to a color heat map, etc. For example, a color heat map may be used to indicate areas of tightness, pulling, chafing, etc., when the product is aligned with the body. As previously discussed, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile. At step 235, feedback may be obtained from social networks with which the user is connected or direct third party feedback to assist in modifying the user's shopping profile.

[0066] At step 237, product customization may integrate data from the product fitting feedback at step 230 and / or social feedback at step 235 to help further personalize the product being considered for acquisition according to the digital avatar and the product fitting module 125 and / or the social shopping module 130.

[0067] At step 240, a personalized product may be ordered by the user, whether in a brick-and-mortar store or an online store. Additionally, the personalized product may be ordered from a manufacturer that may manufacture the product based on the user's requirements such that the product is a one-time customized product for the user. Custom products may include various types of customizations, including, for example, material type, print sample, color, size, quantity, angle, model variation, style, and personalization.

[0068] Reference is now made to FIG. 3, which is a flow diagram for facilitating a personalized offline (in-store) shopping experience, according to some embodiments. As shown in the figure, product information from a product database 300 may be used to discover, purchase, or manufacture products by online users in a computing system supporting a back-end or brick-and-mortar store. In some embodiments, the product database is associated with a product data processing module, which is adapted to perform high intensity calculations on a local POS device and / or on the cloud, depending on the embodiment type and requirements. At step 305, historical data for the user may be obtained, for example, based on previous user purchases at a store or chain store. At step 310, user preference data, such as size, color, material, type preferences, etc. may be obtained. At step 315, scanned or graphic data may be obtained for the user at the front-end or user side, for example, from standard photographs, 2D and / or 3D image scanning, or from capture and processing using a POS vending machine or device, etc. In some embodiments, dedicated or general-purpose applications on a smartphone, tablet, or other computing device may be used to enable effective photographing or scanning of the user. In further embodiments, dedicated or general purpose cameras or scanning devices, vending machines or standing stations (mobile or fixed) may be used by shopping assistants, helpers, salespeople and / or associates. At step 320, this geometric data, along with various input data from steps 305 and 310, is used by the personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. In some cases, the profile may be initiated, for example, by an end user device, a web server, or the like.

[0069] In step 325, a matchmaking of the user shopping profile to the researched or requested products is performed. In this step, product data from the product database 300 is matched with the products requested by the user according to the particular user shopping profile. The matching step can be complemented by providing recommendations to the user based on the aforementioned user profile to product matchmaking process, thereby enabling a high degree of filtering out products that are not suitable for the particular user and a high degree of matching suitable products according to the particular user's personal shopping profile and preferences. This high degree of filtering allows, for example, a store salesperson, or the user himself, to be presented with products that are sufficiently suitable, optionally products currently available, rather than user selected items that are not suitable and thus waste the time and resources of the shopping assistant and the shopper himself. This also allows the user to benefit, optionally in a de-identified manner, from matching and recommendation data generated for other avatars or users that may share similar characteristics, thus enabling smarter and more accurate matching and / or recommendations.

[0070] At step 330, product fitting data from physically present personnel or remote people's feedback may be used to help correct the matchmaking of user data to product data, e.g., feedback from a salesperson in the store or from remote people connected, e.g., via a smartphone or computer, may be used to update the user profile. In some cases, feedback from a salesperson or friend, e.g., which color looks good or which size looks best, may be used by the user to update his or her shopping profile. At step 335, feedback may be obtained from the user using active and / or passive approaches. For example, active input of feedback may occur when the system receives actual feedback regarding fit (e.g., how good / bad / unsatisfactory) or other aspects, whether from a person or people present and / or remote people. Such feedback may allow the user, for example, to input their choice into the system, e.g., via a box or text input element in which the user may enter a selected size, type, or other preference. Passive feedback may occur when the system receives actual feedback regarding fit or other aspects resulting from sales information, returns, etc., or from trying on a type, color, size, etc., allowing the system to learn from the user's past choices and behavior to further improve the personal avatar and product information and fit for other users. In some cases, advanced graphics processing and 3D rendering may be used for the user to try on the product being researched, so that the user may see themselves virtually wearing the product according to a digital simulation that places the product on the user's shopping avatar. As previously mentioned, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile.

[0071] In step 340, a personalized product may be ordered by the user from a brick-and-mortar store, for example for a product that is in principle available but not currently in-store, or to enable production of a specifically requested product based on the user's shopping avatar such that the product is a one-time customized product for the user.

[0072] At step 345, the user can select a personalized product to purchase a custom product created and / or manufactured for her / him, and / or optionally modify or design the product based on products she / he has seen in the store and selected as preferred. These modifications may include visual modifications such as personalization, color, material, printing, and physical characteristics such as control of heel height for shoes, frame thickness for eyewear, and the like. At steps 340 and 345, these features may enable the in-store customer to enjoy features typically limited to e-commerce and online shopping.

[0073] Reference is now made to FIG. 4, which is a flow diagram for facilitating a personalized online shopping experience, according to some embodiments. As shown in the figure, an online store may obtain product information from a product database 400 for selection of products to offer to an online user. At step 405, historical data for the online user may be obtained, for example, based on previous user purchases and research. At step 410, user preference data, such as size, color, material, type preferences, etc. may be obtained. At step 415, scanned or graphic data may be obtained for the user, for example, from standard photographs, 2D and / or 3D image scanning, or from capture and processing using a POS vending machine or device, etc. In some embodiments, dedicated or general-purpose applications on a smartphone, tablet, or other computing device may be used to enable effective photographing or scanning of a user. In other embodiments, a webcam, 3D camera, video recorder, etc. may be used to obtain the scanned or graphic data. At step 420, this graphical data, along with various input data from steps 405, 410 and 415, is used by the personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data.

[0074] Matchmaking of the user shopping profile to the products being researched or requested is performed in step 425. In this step, product data from the product database 400 is matched to the products being researched by the user according to the particular user shopping profile, thereby enabling, for example, advanced product match recommendations to help eliminate products that are not suitable for a particular user, as well as advanced matching of suitable products according to the particular user's personal shopping profile and preferences.

[0075] At step 430, product fitting data from feedback of remote people, such as family, friends or shopping assistants connected via smartphone or computer, may be used to help modify the matchmaking of the user data to the product data, and may be used by the user to update his shopping profile to include data related to, for example, what colors look good or what sizes look best. In a further example, the code may be used to provide product color recommendations, size or fit recommendations, and the like. This feedback may be collected proactively from the user or statically, for example, based on purchase information, shipping and return data, and the like. At step 435, feedback may be obtained from social networks to which the user is connected to help modify the user shopping profile. In addition, human and / or machine-based digital representations of style experts and / or additional guide information may be input to improve the guidance and support provided to the shopper in the online purchasing process. In some cases, advanced graphics processing and 3D rendering may be used to enable the user to virtually try on the product being researched, so that the user may see themselves wearing the product according to a digital simulation that places the product on the user shopping avatar. This may be done using real-time simulation, allowing for a live stream of animated video or high resolution images of the simulation, etc. "Digital try-on" may in some embodiments include physical simulation to include precise positioning of elements on the avatar in motion or in static positions. As previously mentioned, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile.

[0076] At step 440, a product may be ordered by the user from the online store. Optionally, at step 445, a personalized product may be generated by the user from the online store to enable the production of a specially requested product based on the user's shopping avatar so that the product is a one-time customized product for the user. The shopping system may directly connect to the company's manufacturing hardware and ERP system, as needed, in case of custom manufacturing output, to facilitate the production of such personalized products. In one example, the personalized product may be represented in a 3D printer file such as an STL model or a digital cutting machine such as a DXF or DWG file. In other embodiments, this may be a custom routing card or manufacturing instructions and BOM files. Additional inputs may include visual renders to assist the product manufacturer or printer to visually design the custom product.

[0077] In some embodiments, data from product database 400 may be used along with the body or avatar profile derived in step 420 to develop a customized product in step 445, optionally without product matchmaking in step 425.

[0078] Reference is now made to FIG. 5, which is a flow diagram illustrating an example of a personalized hybrid shopping experience, according to some embodiments. As seen in the figure, product information from a product database 500 may be used in a back-end or computing system supporting a brick-and-mortar and / or online store to discover, purchase, or manufacture products by an online or in-store user. At step 505, historical data for the user may be obtained, for example, based on previous user purchases at a store or chain store, whether online and / or in-store experience. At step 510, user preference data may be obtained, such as size, color, material, type preferences, etc. At step 515, scanned or graphic data may be obtained for the user, for example, from standard photo, 2D and / or 3D scanning, or from capture and processing using a POS automated machine, etc., whether performed at the front-end or user side, by the user or by a shopping assistant. In some embodiments, dedicated or general-purpose applications on a smartphone, tablet, or other computing device may be used to enable effective photographing or scanning of the user. In further embodiments, a dedicated or general purpose camera or scanning device may be used by the shopping assistant, which may include a portable or stationary device, a vending machine type or a standalone device. At step 520, this graphical data, together with the various input data from steps 505, 510 and 515, is used by the personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. One advantage of this system is the seamless transition that allows the user to move between online and offline shopping while enjoying the benefits of personalization using a constantly updated personal profile in online and / or in-store scenarios.

[0079] In some embodiments, in step 525, a matchmaking of the user shopping profile to the researched or requested products is performed, optionally for online shopping located in a brick-and-mortar store. In this step, product data from the product database 500 is matched with the products requested by the user according to the particular user shopping profile, thereby enabling advanced elimination of products that are not suitable for the particular user, and advanced matching of suitable products according to the particular user's personal shopping profile and preferences. This advanced filtering allows, for example, a store salesperson, or the user himself, to be presented with fully suitable products, optionally currently available products, rather than user selected items that are not suitable and therefore waste the time and resources of the shopping assistant and / or the shopper himself.

[0080] At step 530, product fitting data from physically present personnel or feedback from remote people may be used to help modify the matchmaking of the user data to the product data, e.g., feedback from a salesperson in the store or from remote people connected, e.g., via a smartphone or computer, may be used to update the user profile. In some cases, feedback from salespersons or friends, e.g., which colors look good or which sizes look best, may be used by the user to update their shopping profile. At step 535, feedback may be obtained from social networks to which the user is connected to help modify the user shopping profile. In some cases, advanced graphics processing and 3D rendering may be used to allow the user to virtually try on the products being researched, so that the user can see themselves wearing the products according to a digital simulation of placing the products on the user shopping avatar. As mentioned above, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile.

[0081] At step 540, a product may be ordered by the online user in the brick-and-mortar store. At step 550, a personalized product may be ordered by the online user in the brick-and-mortar store, for example, for a product that is in principle available but not currently in-store, or to enable production of a specially requested product based on the user's shopping avatar such that the product is a one-time customized product for the user.

[0082] In some embodiments, for a user in a brick-and-mortar store, matchmaking of the user shopping profile to the products being researched or requested is performed in step 545. In this step, product data from the product database 500 is matched with the products requested by the user according to the particular user shopping profile, thereby allowing for advanced filtering of products that are not suitable for the particular user and for advanced matching of suitable products according to the particular user's personal shopping profile and preferences. This advanced filtering allows, for example, a store salesperson, or the user himself, to be presented with fully suitable products, optionally currently available products, rather than user selected items that are not suitable and thus waste the time and resources of the shopping assistant and / or the shopper himself.

[0083] Reference is now made to FIG. 6, which is a flow diagram illustrating an example of personalized shoe shopping, online or offline, according to some embodiments. As shown, insole information from an insole product database may be obtained at 600. Additionally, last information from a last product database may be obtained at 605. In some embodiments, the insole data, last data, and / or shoe model data may include data regarding each product's shape, volume, shoe material, closure type, width, length, height, thickness, material elasticity, comfort fit, and the like. In some examples, mesh analysis and digitization may be performed, for example, on each 3D last in combination with 2D DXF data to be added to the last database 605. Additionally, shoe model data for finding, purchasing, or manufacturing shoes may be obtained from the shoe model database 610. In some cases, for example, a 3D STL file may be imported for a last and a 2D DXF file may be imported for a last sole.

[0084] At step 615, scanned or geometric data may be obtained for the user, for example, from standard photographs, 2D and / or 3D image scanning, or from capture and processing using an automated shopping assistant device or the like. This graphical data may be processed at 620 by the personalized shopping system, across the cloud or within the device itself, to generate a physical profile of the user based on the user's physical characteristics. The profile may include all or some of the following attributes: a 3D mesh containing the exact geometry of the profile, a 3D mesh containing the exact geometry of one or more parts of the user's body, 2D images, characteristics calculated from one or more of the input methods, including specific volumes, cross-sectional measurements, specific distances and lengths, and non-numerical attributes such as preferences. For example, one or both feet of the user may be photographed or scanned together while standing on a floor or another other surface, on or near a reference object, etc. Additionally, one or both feet of the user may be scanned by a camera scanning around different angles of the foot, or by a person moving around the camera, thereby generating a 3D model, video, or series of images as a reference. The scanned data may typically be processed, optionally including interpolation and / or cleaning, to enable object identification and mesh generation, or other suitable processing means. In some cases, meshing may further enable removal of extraneous geometry and / or correction of mesh errors, including, for example, separating, identifying and preparing each of the feet, removing floor, pants or other extraneous material from the scan, etc. Processing of the scanned data may in some cases enable feet alignment, which also assists in separating or individualizing the feet to provide accurate measurements of the feet, including dimensional analysis to determine overall and region-specific length, width and height, as well as cross-sectional analysis to determine area, circumference, and other dimensions at specific cross-sections. Processing of the scanned data may enable smoothing of edges of the scanned feet, formation of missing volumes, construction of the sole of the foot, etc.In some cases, full foot volume and / or region-specific volumes may be extracted from the model as additional information. In step 625, the processed user scan data may be parameterized to extract, for example, accurate length, width, height, arch, ball, cross-section, circumference and volumetric dimensions. These parameters may be calculated from the clean 3D mesh using specific algorithms. For example, the calculation of the arch height of a model foot scanned in a standing position is complex and may be based on a comparison of XYZ parameters of various anatomical sites across different cross-sections at the center of the foot. The foot width may be calculated based on the toe-box volume calculated at both the 3D mesh and 2D cross-section levels. The foot length may be calculated from a combination of the total length and the "ball length", which represents the distance between the heel of the foot and the first metatarsal bone. In addition, user-specific conditions such as pain, infection, injury, etc. may also be identified and integrated into the user's shoe shopping profile. In this step, the pronation or supination state of the foot may also be analyzed. Additionally, the arch of the foot may be identified and measured for fitting of a support insole or other prosthetic device.

[0085] At step 630, historical data for the user may be obtained, for example, based on the user's previous purchases at a store or chain, whether online and / or in-store experience. At step 635, user preference data, such as size, color, material, type preferences, etc. At step 640, a multi-dimensional user shoe shopping profile (hereafter referred to as a user shopping avatar) may be created by processing the various input data from steps 615, 620, 625, 630 and 635, thereby generating a user shoe shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. In some embodiments, the user shoe shopping profile includes both feet of the user, which are typically different and for which parameters are determined separately, thereby benefiting from individual profiles for left and right feet.

[0086] In step 645, matchmaking of the user shoe shopping profile to the shoe products being considered, researched or requested is performed. In this step, product data including dimensions from product databases 600, 605 and 610 are matched to the products being researched by the user according to the particular user shopping profile, thereby allowing advanced elimination of products that are not suitable for the particular user, and advanced matching of suitable products according to the particular user's personal shopping profile and preferences. The matching step may be complemented by providing recommendations to the user based on the aforementioned profile-to-shoe matchmaking process. This step may use the foot model and shoe digital profile directly and / or parameterized numerical models.

[0087] At step 650, product fitting data from physically present personnel or feedback from remote people may be used to help modify the matchmaking of the user data to the product data, e.g., feedback from a salesperson in the store or from remote people connected via, e.g., a smartphone or computer, may be used to update the user profile. In some cases, feedback from salespersons or friends, e.g., which colors look good or which sizes look best, may be used by the user to update their shopping profile. In some cases, advanced graphics processing and 3D rendering may be used for the user to try on the product being researched, so that the user can see themselves virtually wearing the product according to a digital simulation of placing the product on the user shopping avatar. As mentioned above, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile. At step 655, feedback may be obtained from social networks to which the user is connected to help modify the user shopping profile.

[0088] At step 660, the personalized shoes may be ordered by the user in a brick-and-mortar or online store. Additionally, the personalized product may be ordered from a manufacturer that may manufacture the product based on the user's request such that the product is a one-time customized product for the user. Customized footwear according to the present invention may be customized and / or personalized, for example, in one or more of the following ways: shape (e.g., size, length, geometry, volume), design (e.g., color, pattern, print, material) or any other specification or combination of the foregoing.

[0089] Reference is now made to FIG. 7, which is a flow diagram illustrating an example of personalized eyewear shopping and manufacturing, online or offline, according to some embodiments. This embodiment refers to both sunglasses and any type of optical eyeglasses. As shown in the figure, eyeglass model information from an eyeglass product database can be obtained at 700 from an eyeglass model or product database in various file types or data structures. According to some embodiments, eyeglass frame acquisition can be enabled by preparing a parameter model, for example, by generating model parameters for multiple types of inputs, processing resolution adjustments, enabling automatic rigging and skinning, integrating range of motion animations and corrective shape-keys, etc.

[0090] At step 705, historical data for the user may be used, including, for example, facial anatomical landmarks and measured distances, such as, for example, general distances between, the positions of each eye, cheeks, temples, and various points on the ears and nose. General parameters for each face may also be loaded, optionally including volumes, proportions, and standard shapes, and may be obtained, for example, based on previous user inspections and / or purchases at a store or chain store, regardless of online and / or in-store experience. At step 710, user preference data may be obtained, such as size, color, material, type preferences, use needs, etc. This data may be obtained directly or indirectly, for example, using purchase information, questionnaires, forms, and / or any other data acquisition method.

[0091] At step 715, scanned or graphic data may be obtained for the user, for example, from standard photographs, 2D and / or 3D image scanning, or from capture and processing using an automated shopping assistant. This includes any type of 3D scanning technology as described in the previous section or alternative 2D methods that may or may not use a reference object for sizing. Typically, scan data of the user's head and face may be obtained at this stage. In some embodiments, preparation and analysis of the head and face model may include model preparation such as smoothing and cleaning, reconstruction of the mesh with improved topology and / or optimized detail / weight ratio compression for optimal display in real time, and / or orientation and placement. Additionally, facial feature recognition may be used, for example, to generate rendered facial models from multiple angles (e.g., 3-15 angles, standard, depth, to help determine color). Moreover, computer vision and / or machine learning algorithms may be applied to identify eyes, nose, nose bridge, temples, ears, etc. Processing of the scanned data may further include projection of 2D landmarks onto the 3D model, and verification of anatomical landmarks. In the absence of landmarks, an assessment based on statistical or empirical results may be applied to the replacement of these landmarks.

[0092] At step 720, the processed user scan data may be parameterized to extract, for example, accurate face length, width, height, proportions, nose width and volume, ear size, ear height, ear position, skin color, and / or other relevant facial features and dimensions. This data, together with the user history data from 705 and the user preference data at 715, is processed by the personalized shopping system at 725 to generate an eyeglass shopping profile for the user based on the user's facial profile and other physical characteristics, as well as the user behavior and user preference data. In some embodiments, the user eyeglass shopping profile includes both of the user's eyes, which are typically different and thereby benefit from individual profiles for the left and right eyes. In some embodiments, the optical prescription will be collected using photographs, electronic forms, or inspection methods embedded within the application, or in some cases from an external prescription file. According to some embodiments, the system may recommend a particular model based on face shape, previous purchases and history, and optionally based on comparison with similar avatars (optionally anonymous).

[0093] Matchmaking of the user eyewear shopping profile to the products being researched or requested is performed at step 730. At this step, product data from the product database 700 is matched to the products being researched by the user according to the particular user shopping profile, thereby allowing for advanced elimination of products that are not suitable for the particular user and advanced matching of suitable products according to the particular user's personal shopping profile and preferences.

[0094] At step 745, each frame is tried on the user's face using frame customization, which may include, for example: taking key measurements on the subject's or user's face; applying an iterative comparison algorithm to fit the frames to the face; digitally positioning the frames on the subject's face according to the subject's face as per a prepared user eyeglass shopping profile or avatar, properly orienting the frames relative to the face to scale nose bridge size, width and position, adjusting arm folds, arm length and pentoscopic tilt or angle, etc.

[0095] At step 735, product fitting data from physically present personnel or feedback from remote people may be used to help refine the matchmaking of user data to product data, for example feedback from a salesperson or optometrist in the store, or from remote people connected, for example, via a smart phone or computer, may be used to update the user profile. In some cases, feedback from a salesperson or friend, for example, on what colors look good or what size, style, type looks best, etc., may be used by the user to update their shopping profile.

[0096] In some embodiments, advanced graphic processing may be used to allow the user to virtually try on the product being researched, whereby the user may see themselves wearing the glasses according to a digital simulation of placing the glasses on the face of the user's shopping avatar. The virtual try-on may, in some embodiments, include features such as a physical simulation of placing the glasses in the correct or proper position and being able to slide them along the nose. In addition, the try-on may include overlaying a photo or 3D model onto a face model and / or a series of photos or any combination of the foregoing. Animated effects may be included to highlight different attributes of the glasses, including customization animations in the case of custom frames, or other animations such as fly-in / fly-out animations for swapping between glasses. As previously mentioned, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile. In some embodiments, the user may view the customized frames on a digital version of their face to provide meaningful visual feedback. For example, the user's face may be displayed in a 3D viewer and the appearance may be enhanced to provide one or more 3D view manipulations (e.g., zoom, rotate) and animated effects that complement the user experience, such as, for example, a breathing face, smiling, blinking or other animated or static visual effects. Additionally, the user may therefore be provided with customization options, including the ability to select any frame from a collection, customize frame and lens color, customize an auto-recommended fit, personalize files (e.g., text, prescription, etc.), and enable side-by-side comparison of different frames. At step 740, feedback may be obtained from social networks to which the user is connected to assist in modifying the user shopping profile. Needless to say, other steps in the combination of steps may be used to process the input data.

[0097] Step 750 refers to an embodiment where the system can manufacture custom eyewear based on an automated or semi-automated parametric design that fits the frame to the user. In step 750, the associated eyeglass manufacturing print and cut files can be prepared by the system, if necessary. In some embodiments, a standard form 3D printing file, e.g., STL or OBJ, and a 2D lens cutting file, such as DXF, can be prepared. In some embodiments, the system creates two or more paired models for each of the frame designs. This allows, for example, a lightweight model to be used for visualization purposes in the front end of the application, while maintaining a high-resolution model for print file preparation, which may include high-resolution features and details, such as hinges, grooves, angles, temples, etc. Eyewear customization of the printed model described herein can be automatic and / or manual. Additionally, file preparation for 3D printing can include automatically repairing printability issues, generating standards, duplicates, holes and non-manifold geometries, etc. In some embodiments, the system may create custom tags, labels, or other product identification or recognition technologies or devices on glasses or eyewear, for example, including text, QR or bar codes that enable traceability throughout the manufacturing and sales process.

[0098] At step 755, personalized glasses may be ordered by the user, whether in a brick-and-mortar store or an online store. Additionally, personalized products may be requested from a store or ordered from a manufacturer, which may manufacture the product based on the user's request such that the product is a one-time customized product for the user.

[0099] According to an embodiment, a system and process is described for automated personalized product ordering using a digital mirror or personalized viewing protocol. This embodiment integrates virtual and / or augmented reality to manipulate, view and / or try on a specified accessory, such as an existing or designed pair of glasses, on a screen, table, smartphone, communication device, etc., allowing for a visual display of custom or non-custom frames on the client's face.

[0100] According to an embodiment, a file format adapted to enable personalized product ordering is provided that incorporates all of the relevant information, including physical characteristics and personal preferences, to represent a user and assist the user in performing personalized custom or non-custom shopping for clothing, eyewear, or other body-related products. This avatar standard format can be used to plug into virtually any shopping platform, online or physical, to enable in-store customization to fit the physical and aesthetic needs and preferences of the customer.

[0101] Reference is now made to Figures 8A-8G, which show different views of a POS device, launch pad or vending machine, according to some embodiments. In some embodiments, the shopping assistant device generates a user avatar based on one or more of 3D scanning, image acquisition using one or more cameras, measurement of the user profile using a pressure plate, etc. In some embodiments, a standing pad may be used that serves as a positioning pad or reference on which the user stands to perform the body scan. In some cases, the standing platform may include lighting at the bottom of the standing pad to minimize or eliminate shadows and cause black / white peripheral effects, thereby supporting the acquisition of accurate measurements of any color from the camera or scanner on the device. In some embodiments, there may be a proximity sensor for each foot - measuring when the foot is in place or nearby and telling the user to move if necessary. Additionally, the device may include a distance or proximity sensor / robot that recognizes approaching users to attract them to the device, in which case the device may automatically start when the user enters a selected geographic zone. In further embodiments, a sensor may be used to measure the height of the user's instep - for example, to establish the volume of the instep. In still further embodiments, specialized sensors may be used, for example sensors for measuring diabetic ulcers, etc. Such sensors may be used, such as cameras, 3D sensors, all-around cameras, and / or multi-lens cameras.

[0102] 8A is an exploded view of example components of a POS device or automated vending machine in some embodiments. As shown, the automated vending machine 800 may include a standing base 805, an optional light source for illuminating the standing base, a proximity sensor(s) 816 for identifying the positioning of the foot to be scanned by the automated vending machine 800, a standing pad 815 with space for standing on both feet, and a padding layer 820 on which the foot rests. In some embodiments, the padding layer 820 may integrate a pressure sensing mechanism such as a flatbed scanner; a plate of pins; a pressure plate; a touch screen type surface with capacitors and / or pressure sensor elements, etc., to determine the arch dimension of the foot. In some embodiments, the arch dimension measuring element(s) may be used to determine the need for and / or the dimensions of an insole for the user.

[0103] In some embodiments, the arch dimension measuring element(s) may be used to determine the need for and / or the dimensions of a user's instep profile.

[0104] In some embodiments, the arch dimension measuring element(s) may be used to determine the ball-of-the-foot profile needs and / or dimensions of the user.

[0105] In other embodiments, one or more lasers or other illumination mechanisms may be used to determine the dimensions of the arch, ball, or instep of the foot. In one example, a laser from above the foot may be used to identify the size of a "hidden" area not seen by the laser, for example, and use that "hidden" space to determine parameters such as height at different points on the foot, thereby indicating the height or even the size of the arch and bridge of the foot. In a further example, a diffraction grating, prism, or other filter using multiple lines allows the highest point of the foot to be identified.

[0106] In still further embodiments, one or more, optionally differently colored, lights may be used, separately and / or in combination, along with image processing to neutralize the color of the sock and / or foot to assist in identifying non-foot spaces. In other embodiments, one or more may be identified along with the arch of the foot, for example, using background subtraction techniques.

[0107] Furthermore, the automatic vending device 800 may include a body 822 including a computer holding stand 825 and a camera holding element 826 for holding one or more camera elements 835, a panel element 830, a further panel or cover element 845, a computing screen, preferably a touch screen PC or tablet 840, optionally having a place for setting up a proximity sensor 850, for example for identifying a user in proximity to the computing device.

[0108] FIG. 8B is a front view of an example POS device or vending machine.

[0109] FIG. 8C is an isometric view of an example POS device or vending machine.

[0110] FIG. 8D is an isometric front view of an example POS device or vending machine.

[0111] FIG. 8E is an isometric rear view of an example POS device or vending machine.

[0112] FIG. 8F is a side view of an example POS device or vending machine.

[0113] FIG. 8G is a top view of an example POS device or vending machine.

[0114] 8H is a diagram of one or more example sensors in a POS device or vending machine. As shown in the figure, the sensors may comprise, for example, one or more proximity sensors, and an LED flatbed may be configured to provide a light source from below the standing area.

[0115] 9, there is shown a schematic diagram of a shopping assistant system 900 including an automated shopping assistant device 906 that integrates a computing component adapted to execute a shopping assistant application or software program 915. The automated shopping assistant device 906 is adapted to connect to a communication network, such as a communication cloud 920, to access a user shopping profile 925, if on the cloud, and / or to transmit the user shopping profile to the cloud. Furthermore, remote user mobile devices, such as smartphones, tablets or other camera-supported mobile communication devices 905, 910, are adapted to execute the shopping assistant application or software program 916 according to some embodiments. The devices 905 and 910 typically include one or more cameras 910, allowing for example to capture information about a person's foot or both feet, optionally simultaneously using standard images / scans, videos, a series of images or advanced detection components such as structured illumination in IR / NIR or visible light conditions, time-of-flight or other. The devices 905 / 910 typically include a gyrometer to provide camera orientation data to the device camera, for example, to enable images to be captured only when the camera is substantially flat. In some embodiments, internal detection components and / or additional sensors may be attached to the communication devices 905, 910 to provide supplemental data to the system. For example, such sensors may help improve the accuracy of measurements, capture data with real-time feedback, and / or assist the user while providing input to the calculation engine. The remote devices 905 and 910 may communicate with the communication cloud 920, and in particular may be connected to the device user's digital shopping avatar or profile 925.In some embodiments, users may create shopping profiles for one or more users using mobile devices 905, 910 in addition to or instead of the automated shopping assistant device 906.

[0116] Reference is now made to FIG. 10, which illustrates a schematic diagram of a shopping assistant system 1000 and the workflow between components. As shown in the figure, a launchpad or shopping assistant device 1005 can scan a user to generate a shopping profile for the user. The generated profile is sent to the user's mobile device 1010, which can then be used to scan products, such as shoes 1015, optionally with a product tag, such as a QR code 1020, representing the selected product. Additionally, in some embodiments, a user can build a shopping profile using a mobile application 1025. In some embodiments, the user shopping profile can be used to augment in-store shopping, at 1030. In some embodiments, the user shopping profile can be used to augment online shopping, at 1040.

[0117] Reference is now made to FIG. 11A, which is a flow diagram illustrating an example of personalized footwear shopping in a store using an in-store or POS automated shopping assistant device in combination with a mobile computing device application. As shown in the figure, at step 1100, a user may be identified by a proximity sensor in the shopping assistant device, and at step 1105, the user may be requested to stand on a marked pad, optionally providing interactive guidance to ensure that the user stands in the correct position to enable accurate scanning. At step 1110, the device scans the body or body part / element, and at step 1115, generates a shopping profile based on the scan. At step 1120, the device may present a shopping avatar to the user in a graphic format. At step 1125, the device may transmit the shopping avatar to the user's mobile device. At step 1130, the user may open a shopping application, which the user may use to assist with shopping and / or research. At step 1135, the user may use the avatar on his / her mobile device to shop or research, for example, by scanning selected products using a product identifier or recognition device or technology, for example, a coded tag, QR code, or barcode. In general, the scanned product may be processed in a manner relevant to the user avatar, for example, to determine whether the product fits the avatar, fits the user's profile or preferences, etc. In some cases, the application may provide a graphical simulation of the selected product on the avatar. At step 1140, the application may connect to a communications cloud or other database to match the selected product(s) with product data, for example, to assist in determining purchasing options, availability, product quality, product features, reviews, sizing, etc. In some cases, at step 1145, the application may present the user with recommendations, purchasing data, etc.In a further step, the application may enhance the in-store shopping experience by providing, for example, shopping advice, options, shortcuts, access to additional databases, and the like.

[0118] Reference is now made to FIG. 11B, which is a flow diagram illustrating an example of personalized footwear shopping in a store using an in-store or POS automated shopping assistant device in combination with a mobile computing device application. As shown in the figure, at step 1100, a new or known user may enter on the shopping assistant device, for example, with a biometric identifier, an entry screen, etc. At step 1105, the user may be requested to stand on a marked pad, and optionally provide interactive guidance to ensure that the user stands in the correct position to enable accurate scanning. At step 1110, the device scans the body or body part / element, and at step 1115, processes the scan data and generates a shopping profile based on the scan. At step 1120, the device may present the shopping profile to the user in a graphical or other format, as a simulation, shopping avatar, or other virtual assistant. At step 1125, the device may transmit the shopping avatar to the user's mobile device in a format or configuration usable by the software, code, or application(s) of the device. At step 1130, the user may open a shopping application or other program that may manipulate the shopping avatar with the shopping assistant application. At step 1135, the user may use the avatar on their mobile device to shop or research, for example, by scanning a selected product using a product identifier, such as a coded tag, QR code, or barcode, etc. In general, the scanned product may be processed in a manner that incorporates the generated shopping avatar, for example, to determine whether the product fits the avatar, fits the user's profile or preferences, etc. In some cases, the application may provide a graphical simulation of the selected product on the avatar.At step 1140, the application may connect to a communications cloud or other database to match the selected product(s) with advanced product data to help determine, for example, purchasing options, availability, product quality, product features, reviews, sizing, etc. In some cases, at step 1145, the application may present the user with recommendations, purchasing data, purchasing choices, reviews, news, etc. to help enhance the online store shopping experience. In some cases, at step 1150, the application may present the user with recommendations, purchasing data, purchasing choices, reviews, news, etc. to help enhance the in-store shopping experience, for example, by providing shopping advice, options, shortcuts, access to additional databases, etc.

[0119] In some embodiments, the user shopping experience may be performed for additional users connected to the user of the shopping assistant. In such cases, the user application may include shopping profiles for multiple users, thereby enabling a user of a mobile device to perform shopping for multiple users according to the user's shopping profiles.

[0120] According to some embodiments, mobile and / or user avatars may be shared with other users. For example, a user may access or manage multiple user profiles, for example, in an avatar or profile wallet or holder, with the user's approval. In such cases, the managing user may shop on behalf of other users. For example, a parent may maintain profiles for all of his or her family members, allowing the parent to easily shop online and / or offline for all associated family members.

[0121] According to some embodiments, additional personalization, such as an icon or photo, can be provided for each user mobile shopping avatar or user shopping avatar. Such personalization can be particularly useful for a managing user to handle multiple users or mobile shopping avatars. This information, associated with any identifier, can be stored on a cloud avatar database and associated with the user in any platform she / he uses. For example, if one member of a family scans and saves one or more of her family members' profiles, these can be shared with another member(s) of the family who can now load them into the in-store system or e-commerce website being used and can later use this personalization information. In the case of a website, the output can be personalized according to this personalization data and can include photos or avatars or even 3D models of other users right next to the information provided, reaffirming the user's confidence that the recommendations are personal and based on his or her profile, or other profiles legitimately used by the user.

[0122] According to some embodiments, the online store may include a user shopping virtual assistant that may appear on virtually any web page (whether optimized for mobile phone, desktop, notebook, tablet, wearable, etc.) to provide recommendations, guidelines or other assistance to the user, when appropriate, using a profile plug-in or other digital object. For example, the virtual shopping assistant may show information about different fits for the user profile being used or otherwise assist the user. For example, a user with a European foot size of 41 may be informed while shopping or browsing within a Nike online store that the size that corresponds to the user's foot profile in Nike shoes is European 42 or US size 10.5. Additionally, if the user profile includes preference data, such as preferred color and fit, the shopping virtual assistant may also provide suggestions or guidelines based on the user preferences. For example, at a Nike shoe store, the shopping assistant may suggest to the user to look for options such as athletic shoes in either blue or green, in a US size 10.5.

[0123] In some embodiments, the virtual assistant may direct the user to a page or pages that match the user shopping profile data and preferences. In one embodiment, the profile may navigate the website to sections that are interesting or appropriate for the particular user while avoiding irrelevant pages. In another embodiment, the system may use the personalization information, alone or in combination with additional users, to reorganize the website and create a personalized version of the website that may represent what he / she may be most interested in and what would best suit him or her.

[0124] According to some embodiments, the virtual shopping assistant may enable the rendering of 3D views of the displayed product, and optionally, the personalized product. For example, custom shoes displayed according to a user's shopping profile may be rendered in 3D from all sides and angles to help the user view the product in multiple dimensions.

[0125] According to some embodiments, a virtual fitting module may be provided to enable a shopping avatar to try on the displayed product(s).

[0126] According to some embodiments, the user shopping avatar may be a one-time avatar for a store. In another embodiment, the user shopping avatar may be applicable to a chain of stores. In other embodiments, the user shopping avatar may be applicable to various brands or stores, for example, all owned by a parent corporate entity. In other embodiments, the user shopping avatar may be applicable to any or all stores through a connection to a universal user profile in the cloud.

[0127] Reference is now made to FIGS. 12A-12B, which are example screenshots showing interactive screens of a shopping assistant screen to guide a user to place their feet on marked pads, according to some embodiments.

[0128] Reference is now made to Figures 13A-13B, which are example screenshots showing an interactive guide on a shopping assistant screen or mobile screen to assist a user in defining their profile and contact information, according to some embodiments.

[0129] Reference is now made to Figures 14A-14B, which are example screenshots on a shopping assistant screen or mobile screen showing a simulated rendering of a scanned pair of feet and lower legs, according to some embodiments.

[0130] Reference is now made to Figures 15A-B, which are example screenshots of shopping assistant screens or mobile screens to assist a user in entering behavior-related information that may be used to provide better user-related output.

[0131] In some embodiments, the shopping assistant device may help chain stores generate loyalty online and / or offline by enabling the automated shopping assistant to proactively up-sell, cross-sell, and the like within multiple stores.

[0132] In one embodiment of the present invention, the user shopping experience for footwear may be substantially enhanced by applying the following steps: measuring the user's physical profile on the device 185 to provide the user's standard sizes and modifications for different shoes / brands; obtaining the customer's identity or shopping profile to the customer's mobile computing or communication device; obtaining the customer's identity or shopping profile from the device 185 to a communication cloud.

[0133] In a first user case of the present invention, the user does not have a previous user profile and the shopping assistance process may be implemented as follows: the user typically takes off their shoes and stands on a launch pad or shopping assistant device at the entrance or shopping area of ​​a store. In the current example, a footwear application is described. The device then measures / scans the user's body areas, such as both feet, and the device or cloud network may then process the user data to generate a user shopping avatar. Once generated, the avatar is sent to the user's mobile device running the shopping assistant application, for example, using email, beacon, SMS, QR code, IR beam, etc. The user may then scan a desired shoe, whereby the device is configured to match the desired shoe against the avatar and then try it on to provide the best fit. The device may also provide user-related product information, such as availability, color, size, related shoes, rankings, etc.

[0134] In a second user case of the present invention, a user has multiple devices and the shopping assistance process may be performed as follows: the user typically takes off their shoes and stands on a launch pad or shopping assistant device at the entrance or shopping area of ​​a store. In the current example, a footwear application is described. The device then measures / scans the user's body areas, such as both feet, and the device or cloud network may then process the user data to generate a user shopping avatar. Once generated, the avatar is sent to the user's mobile device running the shopping assistant application, for example, using email, beacon, SMS, QR code, IR beam, etc. The user may then scan a desired shoe, whereby the device is configured to match the desired shoe against the avatar and then try it on to provide the best fit. The device, in the current embodiment, may provide the user with expert or consulting information, thereby acting at least in part as a salesperson. The device may also provide the user with relevant product information, such as availability, color, size, related shoes, rankings, and options to measure athleticism / style / weight, etc.

[0135] In a third user case of the present invention, an enhanced shopping experience is delivered whereby the automated shopping assistance process may incorporate local shoe scanning. In some cases, user data or avatars may be used to cull appropriate reviews / comments from the online world+social feedback+rankings, sales information / history, recommendations, up-selling, cross-selling, etc.

[0136] The foregoing description of embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. It should be apparent to those skilled in the art that numerous modifications, variations, substitutions, changes, and equivalents are possible in light of the above teachings. It is therefore understood that the appended claims are intended to cover all such modifications as fall within the true spirit of the invention.

Claims

1. 1. A personalized shopping assistant system, comprising:

1. A shopping assistant device configured to capture a foot of a user and generate a user shoe profile based on capturing the foot of the user, wherein capturing the foot of the user includes: acquiring at least one image of the user's foot based on one or more interactive prompts instructing the user on a position of the foot; calculating user measurement data for the foot based on the images; generating a 3D mesh model of the foot based on the user measurement data; the user shoe profile includes the 3D mesh model, the 3D mesh model including the user measurement data; A shopping assistant device; a mobile device application configured to apply the user shoe profile in a shoe model database using a mobile device and match one or more products based on the 3D mesh model to the user shoe profile on the mobile device; The shopping assistant device is configured to transmit the user shoe profile to the mobile device of the user and provide a fit of at least one selected product on the user shoe profile by a user selection of a product from one or more products matched to the user shoe profile; the shopping assistant device is further configured to receive feedback regarding the fit of the at least one selected product on the user via one or more remote computing devices each associated with another user, and to provide one or more matched products modified based on the feedback. A personalized shopping assistant system.

2. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device is further configured to provide a scanning mat including markings to guide a user to stand properly to enable accurate scanning of the user's foot, and the at least one image is captured when the foot is within the markings.

3. 2. The personalized shopping assistant system of claim 1, wherein generating the user shoe profile includes generating a user shopping avatar based on capturing the foot of the user, and the shopping assistant device is configured to display a digital simulation of a fit of the at least one selected product on the user shopping avatar.

4. 2. The personalized shopping assistant system of claim 1, wherein the shoe model database includes an insole model database including insole information and a shoe last database including shoe last information, and the shopping assistant device is configured to generate at least one custom shoe recommendation for the user based on at least the insole model database and the shoe last database.

5. 2. The personalized shopping assistant system of claim 1, wherein the shoe model database includes insole data, shoe last data, and / or shoe model data, and the shoe model data includes product shape data, volume data, shoe material data, closure type data, shoe type data, width, length, height, thickness, material elasticity data, and / or wearing comfort parameters.

6. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device includes one or more sensors configured to capture a depth of the user's foot, and the shopping assistant device is further configured to perform a dimensional analysis to determine one or more of a length, width, height, or cross-sectional analysis of the foot.

7. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device is further configured to process scanned image data to extract one or more of length, width, height, arch, ball, cross-section, circumference and / or volumetric dimensions associated with the user's foot.

8. 2. The personalized shopping assistant system of claim 1, wherein generating the 3D mesh model includes generating one or more user-specific conditions associated with the foot, the user-specific conditions including one or more of pain, infection, injured area, pronation or supination condition, foot arch parameters, insole measurements, or prosthetic measurements.

9. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device is further configured to generate a customized footwear order based on the user selection of products and based on the 3D mesh model.

10. 1. A method of matching a user with a footwear article, comprising: the shopping assistant device starting to capture the feet of the user using one or more integrated imaging sensors; the shopping assistant device providing one or more interactive prompts instructing the user on positioning the foot to guide the user and enable accurate scanning of the foot; said shopping assistant device acquiring at least one image of said foot using said one or more integrated imaging sensors; calculating, by the shopping assistant device, user measurement data of the foot based on the images; generating a 3D mesh model of the foot based on the user measurement data, the 3D mesh model including the user measurement data, by the shopping assistant device; generating a user shoe profile for the user, the shopping assistant device generating a user shoe profile for the user, the user shoe profile including the 3D mesh model of the foot; the shopping assistant device transmitting the user shoe profile to a user mobile device; providing the user with automated product matching for the one or more footwear products based on the 3D mesh model, the shopping assistant device matching the one or more footwear products to the user shoe profile on the user mobile device; providing, based on a user selection of at least one selected product from one or more products matched to the user shoe profile, a fit of the at least one selected product on the user shoe profile; receiving, by the shopping assistant device, via one or more remote computing devices each associated with another user, feedback regarding the fit of the at least one selected product on the user; and providing one or more matched products modified based on the feedback. method.

11. 11. The method of claim 10, further comprising providing a scanning mat including markings to guide the user to stand properly to enable accurate scanning of the user's foot, and wherein the at least one image is captured when the foot is within the markings.

12. 11. The method of claim 10, wherein generating the user shoe profile includes generating a user shopping avatar based on capturing the foot of the user, and the shopping assistant device is further configured to display a digital simulation of a fit of the at least one selected product on the user shopping avatar.

13. 11. The method of claim 10, further comprising applying the user shoe profile in a shoe model database to match one or more products to the user shoe profile based on the 3D mesh model, the shoe model database including an insole model database including insole information and a shoe last database including shoe last information, the method further comprising generating at least one custom shoe recommendation for the user based on at least the insole model database and the shoe last database.

14. 14. The method of claim 13, wherein the shoe model database includes insole data, shoe last data, and / or shoe model data, the shoe model data including product shape data, volume data, shoe material data, closure type data, shoe type data, width, length, height, thickness, material elasticity data, and / or comfort parameters.

15. 11. The method of claim 10, further comprising: capturing a depth of the user's foot using one or more sensors; and performing a dimensional analysis to determine one or more of a length, width, height, or cross-sectional analysis of the foot.

16. 11. The method of claim 10, further comprising processing the scanned image data to extract one or more of length, width, height, arch, ball, cross-section, circumference and / or volumetric dimensions associated with the foot of the user.

17. 11. The method of claim 10, wherein generating the 3D mesh model includes generating one or more user specific conditions associated with the foot, the user specific conditions including one or more of pain, infection, injured area, pronation or supination condition, foot arch parameters, insole measurements, or prosthetic measurements.

18. The method of claim 10 , further comprising generating a customized footwear order based on the user selection of products and based on the 3D mesh model.

Images