System, platform, and method for personalized shopping using automated shopping assistant

Abstract

To provide an apparatus, a system, and a method for personalized online product fitting.SOLUTION: A shopping assistant system 1000 comprises: a point-of-sale based automated shopping assistant apparatus 1005 for generating a 3D user shopping profile based on a capture of at least a part of a user's body; a shopping assistant server connected to a communication cloud-based shopping assistant database and configured to upload the profile to the shopping assistant database; a mobile application 1025 for applying the profile in-store using a user's mobile device; and a plurality of product tags 1020 including scannable product data that is stored on the shopping assistant database to capture products selected by the user using the mobile application.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims priority from U.S. Provisional Patent Application No. 62443275, filed on Jan. 6, 2017, entitled "A SYSTEM, PLATFORM AND METHOD FOR PERSONALIZED SHOPPING USING AN AUTOMATED SHOPPING ASSISTANT", which is hereby incorporated by reference in its entirety.

[0002] The present invention generally relates to methods, applications, and devices useful in personalized product shopping.

Background Art

[0003] The majority of today's shoe shopping still takes place in traditional stores. Most customers are accustomed to the limitations of traditional shopping, such as current store managers and salespersons. Generally, customers are guided towards the products and rely on salespersons for assistance in locating in - stock products, trying on products, etc.

[0004] Furthermore, in a typical shopping experience, customers are substantially required to repeat each visit to the same or different stores, which is highly inefficient and a source of user dissatisfaction.

[0005] It would be highly advantageous to have a system or method that enables highly accurate and user - friendly automated or semi - automated fitting solutions both online and in - store.

Summary of the Invention

Means for Solving the Problems

[0006] According to one embodiment of the present invention, an apparatus, system, and method are provided for providing personalized online product fitting according to several embodiments.

[0007] A method for personalized shopping has steps where an automatic shopping assistant system accesses product data, a matchmaking system accesses user history data, the matchmaking system accesses user preference data, the matchmaking system accesses user anatomical data obtained from an automatic shopping assistant device, and the automatic shopping assistant system matches user history, preference, and anatomical data with product data to generate a personalized matching system.

[0008] In some embodiments, the automatic shopping assistant system matches user history and preference data with product data to generate personalized product recommendations.

[0009] In some embodiments, the automatic shopping assistant system can be used to enable a user to order personalized products.

[0010] In some embodiments, the automatic shopping assistant system can be used to provide a simulation representing one or more anatomical features of a user.

[0011] In one embodiment, a user and one or more third parties can provide product fitting feedback, and a user and one or more third parties can provide social feedback. The automatic shopping assistant system can adjust personalized products based on the product feedback. The system can also provide anatomical data regarding the user, and the automatic shopping assistant system generates a user shopping avatar including one or more characteristics of the user considering the anatomical data.

[0012] The automated shopping assistant system may include virtual try-on features. In one embodiment, before the user orders a personalized product, the product is generated based on an avatar. User preferences can be selected from the group consisting of size, color, material, and type preferences.

[0013] According to some embodiments, a platform for personalized shopping is provided, including a profile module for generating digital avatars for a plurality of users, a product module for integrating product data for a plurality of products, and a match-making module adapted to execute code for matching digital avatar data and product data to generate product recommendations; an end-user computing device communicatively connected to the cloud-based server and including image augmentation elements, wherein the match-making module executes a software application to generate a user mobile shopping avatar based on capturing at least a part of the user's anatomical structure, which is used in generating anatomical data for the digital avatar profile for the user.

[0014] In some embodiments, the platform is adapted to generate and / or present a simulation representing one or more anatomical features of the user.

[0015] In some embodiments, the platform may have a product ordering module, a product customization module, a social shopping module, and / or a product fitting module, etc.

[0016] A handheld system for personalized shopping may have a screen configured to receive user input, a device camera for capturing standard images of the user's anatomical structure, and a processor having a register adapted to analyze anatomical data, product data, user history data, and user preference data. The processor may obtain information from the register, write information to the register, and is configured to match user history and preference data with product data to generate a personalized matching system. The processor is configured to match user history and preference data with product data to generate personalized products, and the user may purchase the personalized products by providing user input to the screen.

[0017] In one embodiment, the system has a depth sensor configured to accurately determine the distance of an object from the sensor, and the depth sensor information is used to match preference data with product data. In some embodiments, the depth sensor enables a 3D scan of at least a part of the body related to the user's anatomical profile and enables capturing the length, width, and depth of that part of the body.

[0018] In some embodiments, the handheld system includes a software application running on the handheld system to generate and present a graphic simulation of the user mobile shopping profile based on the capture of at least a part of the user's anatomical structure.

[0019] A shopping assistant system for shopping using a shopping profile is provided, including a POS-based automatic shopping assistant device for generating a user shopping profile; a mobile device app for applying the user shopping profile in the store; a shopping assistant server connected to a shopping assistant database based on a communication cloud; and a plurality of product recognition devices, such as tags, for enabling the capture of products selected by the user.

[0020] The shopping assistant device may include one or more image scanners for capturing at least a part of the body related to the user's anatomical profile and enabling the capture of the length, width, and depth of that part of the body.

[0021] The shopping assistant may include a product module for integrating product data for a plurality of products, as well as a match-making module adapted to execute code connecting the shopping profile and the product data to generate product recommendations.

[0022] The shopping assistant may include order tags associated with products for the user to place an order by capturing the order tags.

[0023] The shopping assistant may include a proximity sensor associated with the shopping assistant device for identifying the user when the user enters a selected geographical area around the shopping assistant device.

[0024] The shopping assistant may include a proximity sensor associated with the shopping assistant device for enabling an automatic trigger for recognition and scanning when the user is properly positioned near the proximity sensor.

[0025] The shopping assistant may include a virtual try-on module for digitally trying on selected products on the user's shopping profile.

[0026] A method for enhancing in-store shopping is provided: identifying a user entering a shopping area using a proximity sensor associated with an automated shopping assistant device; connecting the user to an automated shopping assistant system to open a user profile; starting to capture one or more body parts by the automated shopping assistant device using one or more image sensors; generating a 3D shopping profile of the user; starting to download an automated shopping assistant application onto the user's mobile computing device; connecting the automated shopping assistant device to the user's mobile device to close the user shopping profile on the user's mobile device; selecting a product of interest to the user by capturing a product tab using the user's mobile device; and providing product-related information for the selected product to the user.

[0027] The method may further include ordering the selected product using the user's mobile device.

[0028] The method may further include customizing the selected product using the user's mobile device.

[0029] The method may further include ordering the customized product using the user's mobile device.

[0030] The method may further include providing product inventory information to the user.

[0031] The method may further include generating a user profile avatar. The method may further include trying on the selected product on the user shopping profile avatar.

[0032] The method may further include sending a tried-on user profile avatar to a social media system for viewing by selected peers to whom the user is connected. The method may further include enhancing a user shopping profile based on user behavior.

[0033] The method may further include providing shopping recommendations based on a user shopping profile and / or user shopping behavior.

[0034] The method may further include using a user shopping profile for shopping at an online store.

[0035] The method may further include enhancing a user shopping profile using an additional imaging device.

[0036] The method may further include generating an additional user shopping profile using a mobile device.

[0037] The method may further include converting a user's profile data, with the user's consent, from an in-store device to a user device, such as a smartphone application.

[0038] A method for personalizing shopping is provided herein: an automated shopping assistant system accessing product data; the automated shopping assistant system accessing user history data; the automated shopping assistant system accessing user preference data; the automated shopping assistant system accessing anatomical data regarding the user and a match-making system executing a software application to generate a user mobile shopping avatar based on the capture of at least a portion of the user's anatomical structure; and the automated shopp The step of a ping assistant system matching user history data, preference data, shopping avatars, and product data to generate a personalized matching system; is included.

[0039] The method may further include the step of generating personalized product recommendations.

[0040] The method may further include the step of ordering personalized products.

[0041] The method may further include the step of a user and one or more third parties providing product feedback.

[0042] The method may further include the step of a user and one or more third parties providing social feedback.

[0043] The method may further include the step of an automatic shopping assistant system adjusting personalized products based on product feedback.

[0044] The method may further include the step of providing a simulation representing one or more anatomical features of a user.

[0045] The method may further include virtual try-on features.

[0046] The method may further include ordering personalized products, where the products are generated based on a user shopping avatar.

[0047] The method may further include selecting a product type from the group consisting of size, color, material, and type preferences.

[0048] The principles and operations of the systems, devices, and methods according to the present invention can be better understood with reference to the drawings and the following description, and it is understood that these drawings are provided for illustrative purposes only and are not intended to be limiting.

Brief Description of the Drawings

[0049]

Figure 1A

Figure 1B

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8A

Figure 8B

Figure 8C

Figure 8D

Figure 8E

Figure 8F

Figure 8G

Figure 8H

Figure 9

Figure 10

Figure 11A

Figure 11B

Figure 12A

Figure 12B

Figure 13A

Figure 13B

Figure 14A

Figure 14B

Figure 15A

Figure 15B

DETAILED DESCRIPTION OF THE INVENTION

[0050] The following description is presented to enable a person skilled in the art to make and use the invention as provided in the context of a particular application and its requirements. Various modifications to the described embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments. Accordingly, the invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the invention.

[0051] As used herein, the term "fitting" refers to trying on a product, viewing where the product has been tried on, and modifying the product to fit a particular person's body or other physical parameters. The term "avatar" refers, in particular, to an embodiment, anthropomorphization, icon, model, or image representing a particular person for representing a person on a screen.

[0052] Relatively low-rate footwear purchase online shopping can be enhanced by optionally providing customers with accurate size recommendations for each shoe model based on a simulation of the customer's foot or an avatar model, resulting in higher customer confidence in purchases.

[0053] Non-limiting embodiments of the present invention include systems, platforms, and methods for facilitating advanced personalized shopping, including effective product fitting, whether online and / or in-store. In some embodiments, systems, platforms, and methods are provided for enabling personalized manufacturing of products.

[0054] Referring now to FIG. 1A, which is a schematic system diagram showing a system 100 for facilitating personalized shopping according to some embodiments. System 100 enables seamless shopping in-store and / or online using highly accurate user shopping profiles and / or avatars generated by an automated shopping assistant device. In some cases, data processing is performed on a cloud and / or on a local automated shopping assistant device.

[0055] As shown in the figure, the personalized shopping system 100 includes a platform 105 for managing personalized shopping profiles, which may include a digital avatar profile module 110, a digital product file module 115, a product selection module 120, a product fitting module 125, a social shopping module 130, a product ordering module 135, and a product customization module 137.

[0056] The platform 105 communicates with a communication cloud 140, which may include a physical profile data module 145 communicatively connected to a start pad, a kiosk, or an automated shopping assistant device 185, which is further communicatively connected to a user 180 to provide physical user data from, for example, 2D and / or 3D scans or other digital measurement sources. The communication cloud 140 may further include a user preference data module 150 communicatively connected to the automated shopping assistant device 185 and / or to 180 to provide user preference data. The communication cloud 140 may further include a product file data module 160 communicatively connected to a product database 165 and a product matching data module 170 including a product matching algorithm communicatively connected to a product database 175.

[0057] In one embodiment, the device 185 includes one or more imaging devices, such as 2D and / or 3D cameras, which may be movable or fixed components. The device 185 may further include one or more sensors, such as proximity sensors, scanners, cameras, pressure plates, and / or other sensors.

[0058] As shown in the figure, the digital avatar profile 110 is highly personalized and constructed from various data sources, whether directly or indirectly from the user and regardless of whether it represents physical characteristics and / or mental, emotional, psychological characteristics. The digital avatar profile 120 generally includes a file or group of files and data points for which instructions can be executed to enable the generation of a high-resolution user profile or avatar from one or more data sources. Further, the product selection module 120 generally matches the personalized avatar profile 110 with the selected digital product, whether online or offline. Calculations for the matching algorithm can be performed by a processor through the storage, retrieval, and processing of data. In some embodiments, one or more device registers may be used, where a register (s) refers to one of a small set of data holding locations that are part of a computer processor for holding computer instructions, memory addresses, or any kind of data. The matching algorithm, in some embodiments, may include executing code to provide a perfect or near-perfect match between product types, sizes, styles, etc., provided by a product database and physical parameters and / or user preference data as defined by a user avatar profile based on a scan of at least a part of the user's body related to the user's profile. For example, User A may have a foot profile defined by size, width, and depth. Further, the user's profile may include preference data for a preferred style, type of shoe, and color, such as blue or gray sports shoes. The product database may include, for example, sports shoes in the colors blue, gray, and blue-gray that fit or are nearly sufficient for the user's size, width, and depth. In this case, the matching algorithm matches the profile definition with one or more products from the product database that match the user profile.

[0059] The product selection module 120 can be further improved by providing system feedback and product fitting data using the product fitting module 125, as well as social shopping data from the social shopping module 130. The product fitting module, in some embodiments, includes means for feedback from a shopping assistant or supporter with the shopper. In other embodiments, the product fitting module includes means for feedback from a virtual shopping assistant or supporter, such as a communicatively connected supporter, a digital or virtual mirror or screen that displays the user with the product. Further, the product customization module 137 can receive data from the product fitting module 125 and / or the social shopping module 130 to assist in further personalizing the product under consideration for acquisition, in accordance with the digital avatar and the product fitting module 125 and / or the social shopping module 130. The product customization module 137 can enable the user to change or customize the product being tried on or tested, for example, by changing the color, shape, design, size, material, etc. of the product. In this way, the ordered product can be constructed according to user-specific or customized requirements. Further, the product customization module 137 can send the customized product as selected or generated by the user to the product selection module 120, and then the product selection module 120 can initiate an order for the customized product via the product order module 135. Moreover, the user update embodied in the user's changes made in the product selection module 120 can be used to update the digital avatar profile 110, thereby keeping the user avatar profile up-to-date with respect to, for example, the user's physical changes, changes in preferences, etc.

[0060] The product selection module 120 includes files or groups of files and data points that can execute instructions to execute commands for matching a high-resolution user profile or avatar with products that highly match the shopping research being conducted by each system user. This module can further integrate the feedback generated in the system module to continuously improve the accurate product recommendations it provides. Using various technical procedures executed on the avatar, such as integrating not only length, width, height, and additional distances, but also volume, cross-sectional area, and outer perimeter, the system can represent the number of avatars in an array. In some embodiments, this array of numbers can represent various elements of the avatar, enabling comparison with similar elements of the products tried on by the avatar. Accordingly, when avatar data, which can be enhanced using, for example, recommendation algorithms and machine learning techniques, is compared with product data from product files, this can enable accurate and continuously improved personalized product recommendations from the system.

[0061] According to some embodiments, a digital avatar profile can be generated with relatively low-level integration of a 3D scanner. In some examples, the sensors or scanners that can be used can include structured light, time-of-light, photogrammetry, or any other type of 3D and / or 2D scanning technology. Suppliers of such technologies include, but are not limited to, PrimeSense (trademark)-based scanners, Occipital Structure Sensor, 3D-Systems Sense and iSense (trademark) sensors, Intel (trademark) RealSense sensors (standalone or integrated into machines), iPad (registered trademark) or tablet-based scanning platforms, PCs (integrated and external), Android+RealSense (next generation) devices, and Google Project Tango devices.

[0062] Referring now to FIG. 1B, which is a schematic system diagram showing a platform 180 for facilitating personalized shopping in accordance with some embodiments. As shown in the figure, the personalized shopping platform 180 includes one or more user mobile devices 182, such as, without limitation, a smartphone or a tablet, including a camera, applications, and a data connection; a store or point-of-sale management device, a vending machine, or an automated shopping assistant device 184, typically located in or near the store, such as an electronic device equipped with one or more cameras, sensors or scanning devices, applications, and a data connection; and the mobile device(s) 182 and / or the automated shopping assistant device 185 are connected to a communication cloud 186. The communication cloud 186 includes a digital product module 190 for holding, integrating, and otherwise managing data for a plurality of products; a digital avatar profile module 188 for holding, integrating, processing, and otherwise managing profile data for a plurality of users; and a matching module 192 for matching product and avatar profile data to assist in enabling product recommendations and other matching functions. The platform 180 further includes a product customization module 194 for enabling the ordering of customized products based on the matching module output and / or user selection; and a product ordering module 196 for enabling the ordering of products based on the matching module output and / or user selection.

[0063] Referring now to FIG. 2, which is a flow diagram showing a process for facilitating personalized shopping, whether online or offline, according to some embodiments. As shown in the figure, product information from product database 200 can be used to discover, purchase, or manufacture products. At step 205, historical data for the user can be obtained, for example, based on previous user purchases and research. At step 210, user preference data such as size, color, material, type preference, etc. can be obtained. At step 215, scanned data or graphic data can be obtained for the user, for example, from standard photographs, 2D and / or 3D image scanning, or capture and processing using a POS self-checkout or automated shopping assistant device. This graphic data is used by the personalized shopping system to generate a user's body profile based on the user's physical characteristics. At step 220, a multi-dimensional user shopping profile (hereinafter referred to as a user shopping avatar) can be developed by processing various input data from steps 205, 210, and 215, thereby generating a user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. The profile or avatar can be an optionally dynamic structure that can be continuously improved using feedback and additional inputs from steps 205, 210, and / or 215 in a way that animates, reflects, or represents the user. In one embodiment, the user avatar can be used to match potential products within the user to those in a network or partnership of online and / or offline stores or platforms, within any retail store within a chain store, or online, within a single store, and / or within multiple stores.

[0064] In step 225, a matchmaking for the products of the user shopping profile that are being researched or requested is performed. In this step, product data from the product database 200 is matched with the products being researched by the user according to a specific user shopping profile, thereby highly removing products that are not suitable for a specific user and enabling highly matching suitable products according to the personal shopping profile and preferences of the specific user. The matching step can be complemented by providing recommendations to the user based on the above-described product matchmaking process of the user profile. In some embodiments, a match is provided between the generated user profile and a plurality of products that match one or more elements of the generated profile, thereby enabling, for example, fit and size recommendations as well as additional data connecting the profile and the product(s).

[0065] In step 230, product fitting data from physical attendants or feedback from people far away can be used to assist in modifying the matchmaking of user data to product data. For example, feedback from in-store salespersons, or feedback from people far away connected via, for example, smartphones or computers, can be used to update the user profile. In some cases, feedback from salespersons or friends, such as which color looks good or which size looks the best, can be used by the user to update their shopping profile. In some cases, advanced graphic processing and 3D rendering can be used for the user to virtually try on the product being researched, so that the user can see themselves wearing the product according to a digital simulation that places the product on the user's shopping avatar. In some cases, the system can provide a static or dynamic high-resolution visual output, such as an animated avatar or character, optionally with a recommended rendered image and / or virtual representation tried on the avatar. For example, such a rendering can enable the user to see the product being worn depicted on the avatar, thereby assisting the user in visualizing details such as fit, tightness, color, style, material, etc. according to, for example, a color heat map. For example, a color heat map can be used to indicate areas of tightness, stretch, rubbing, etc. when the product is aligned with the body. As described above, the user can use the shopping avatar to provide further feedback to modify the user's shopping profile. In step 235, feedback can be obtained from the social network the user is connected to or directly from third-party feedback to assist in modifying the user's shopping profile.

[0066] In step 237, product customization may assist in further personalizing the product under consideration for acquisition by integrating data from the product fitting feedback in step 230 and / or the social feedback in step 235, in accordance with the digital avatar as well as the product fitting module 125 and / or the social shopping module 130.

[0067] In step 240, the personalized product may be ordered by the user, whether in a physical store or an online store. Further, the personalized product may be ordered from a manufacturer that can manufacture the product based on the user's requirements such that the product is a one-time customized product for the user. The custom product may include various types of customizations, including, for example, material type, print sample, color, size, quantity, angle, model variation, style, made-to-measure.

[0068] Referring now to FIG. 3, which is a flowchart for facilitating a personalized offline (in-store) shopping experience, according to some embodiments. As shown in the figure, in a computing system that supports a backend or a physical store, product information from a product database 300 may be used to discover, purchase, or manufacture products by an online user. In some embodiments, the product database is associated with a product data processing module, which is adapted to perform high intensity calculations on a local POS device and / or in the cloud, depending on the embodiment type and requirements. At step 305, historical data for the user may be obtained, for example, based on previous user purchases at a store or a chain of stores. At step 310, user preference data, such as size, color, material, type preference, etc., may be obtained. At step 315, scanned data or graphic data may be obtained for the user at the front end or user side, for example, from standard photographs, 2D and / or 3D image scanning, or capture and processing using a POS self-service machine or device. In some embodiments, a dedicated or general-purpose application on a smartphone, tablet, or other computing device may be used to enable effective photography or scanning of the user. In further embodiments, a dedicated or general-purpose camera or scanning device, a self-service machine, or a standing station (movable or fixed) may be used by a shopping assistant, helper, salesperson, and / or associate. At step 320, this geometric data, along with the various input data from steps 305 and 310, is used by a personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. In some cases, the profile may be launched, for example, by an end-user device, a web server, etc.

[0069] In step 325, a matchmaking of the user shopping profile against the researched or requested products is performed. In this step, product data from the product database 300 is matched with the products requested by the user according to a specific user shopping profile. The matching step may be complemented by providing recommendations to the user based on the aforementioned product matchmaking process for the user profile, thereby highly removing products inappropriate for a specific user and enabling highly matching appropriate products according to the personal shopping profile and preferences of the specific user. This high-level screening enables, for example, a store salesperson or the user himself / herself to be presented with products that are sufficiently appropriate, optionally not appropriate and thus a waste of the shopping assistant's and the shopper's own time and resources, rather than the user-selected items, and currently available products. This also enables the user to benefit, optionally in an anonymized manner, from the matching and recommendation data generated for other avatars or users who may share similar characteristics, thus enabling higher performance and more accurate matching and / or recommendations.

[0070] In step 330, product fitting data from the feedback of physically present personnel or people far away can be used to assist in modifying the matchmaking of the user data to the product data. For example, feedback from in-store salespersons, or feedback from people far away connected via, for example, smartphones or computers can be used to update the user profile. In some cases, feedback from salespersons or friends, such as which color looks good or which size looks the best, can be used by the user to update their shopping profile. In step 335, feedback can be obtained from the user using positive and / or negative approaches. For example, when the system receives actual feedback regarding fit (e.g., better / worse / unsatisfactory in degree) or other aspects, whether from present person or people and / or people far away, positive input of feedback can occur. Such feedback can enable the user to input, for example, the selected option, such as the selected size, type or other preferences, into the system via a box or text input element where the user can input. When the system receives actual feedback regarding fit or other aspects resulting from sales information, returns, etc., or by trying on a certain type, color, size, etc., passive feedback can occur, enabling the system to learn from the user's past selections and behaviors and further improve the personal avatar as well as the fit of product information and to other users. In some cases, advanced graphic processing and 3D rendering can be used for the user to try on the product being researched, whereby the user can virtually see themselves wearing the product according to the digital simulation that places the product on the user shopping avatar. As described above, the user can use the shopping avatar to provide further feedback to modify the user's shopping profile.

[0071] In step 340, the personalized product can be ordered to enable the manufacture of a product that is typically available to the user from a physical store but is currently out of stock, or a product that is specially requested based on the user's shopping avatar such that the product is a one-time customization product for the user.

[0072] In step 345, the user can select a personalized product to purchase a custom product made and / or manufactured for her / him and / or, optionally, modify or design the product based on a product she / he has seen and favorably selected in the store. These modifications can include visual changes such as naming, color, material, printing, etc., and physical characteristics such as control of the heel height of shoes, the thickness of the frame of eyewear, etc. In steps 340 and 345, these features can enable in-store customers to enjoy features that are typically limited to e-commerce and online shopping.

[0073] Referring now to FIG. 4, which is a flowchart for facilitating a personalized online shopping experience in accordance with some embodiments. As shown in the figure, an online store may obtain product information from a product database 400 for selection of products to offer to online users. At step 405, historical data for an online user may be obtained, for example, based on previous user purchases and research. At step 410, user preference data such as size, color, material, type preference, etc. may be obtained. At step 415, scanned data or graphic data may be obtained from a user, for example, from a standard photograph, 2D and / or 3D image scanning, or capture and processing using a POS cash register or device, etc. In some embodiments, a dedicated or general purpose application on a smartphone, tablet or other computing device may be used to enable effective photographing or scanning of the user. In other embodiments, a webcam, 3D camera, video recorder, etc. may be used to obtain the scanned data or graphic data. At step 420, this graphic data, along with the various input data from steps 405, 410 and 415, is used by a personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data.

[0074] At step 425, a match is performed between the user shopping profile and the product being researched or requested. At this step, product data from the product database 400 is matched with the product being researched by the user according to a particular user shopping profile, thereby facilitating, for example, advanced product match recommendations that remove products not appropriate for a particular user, as well as advanced matching of appropriate products according to a particular user's personal shopping profile and preferences.

[0075] In step 430, for example, product fitting data from feedback from people far away, such as family, friends, or a shopping assistant connected via a smartphone or computer, can be used to assist in modifying the matchmaking of user data to product data, and can be used by the user to update their shopping profile, for example, to include data related to which colors look good or which sizes are most flattering. In a further example, code can be used to provide product color recommendations, size or fit recommendations, etc. This feedback can be collected actively from the user or statically, for example, based on purchase information, shipping and return data. In step 435, feedback can be obtained from the social network the user is connected to in order to assist in modifying the user shopping profile. In addition, human and / or machine-based style experts in digital representation as well as / or additional guidance information can be input to improve the guidance and support provided to shoppers in the online purchase process. In some cases, advanced graphic processing and 3D rendering can be used to enable the user to virtually try on the product being researched, whereby the user can see themselves wearing the product according to a digital simulation that places the product on top of the user's shopping avatar. This can be done using real-time simulation, enabling a live stream of an animated video of the simulation or a high-resolution image. "Digital try-on" can include physical simulation in some embodiments to include accurate positioning of elements on the avatar, whether moving or in a static position. As described above, the user can use the shopping avatar to provide further feedback to modify the user's shopping profile.

[0076] In step 440, the product can be ordered by the user from the online store. Optionally, in step 445, a personalized product can be generated by the user from the online store to enable the manufacture of a specially requested product based on the user's shopping avatar such that the product becomes a one-time customized product for the user. The shopping system can directly connect to the company's manufacturing hardware and ERP system, as needed, to facilitate the manufacture of such personalized products in the case of custom manufacturing output. In one example, the personalized product can be represented by a 3D printer file such as an STL model or a digital cutting device such as a DXF or DWG file. In other embodiments, this can be a custom routing card or manufacturing instructions and BOM files. Additional inputs can include a visual render that assists the product manufacturer or printer in visually designing the custom product.

[0077] In some embodiments, data from the product database 400, along with the body or avatar profile derived in step 420, can be used, optionally without a product match in step 425, to develop a customized product in step 445.

[0078] Referring now to FIG. 5, which is a flow diagram showing an example of a personalized hybrid shopping experience according to some embodiments. As seen in the figure, in a computing system that supports a backend or physical store and / or an online store, product information from a product database 500 may be used to discover, purchase, or manufacture products by online or in-store users. At step 505, historical data for the user may be obtained, for example, based on previous user purchases at a store or chain of stores, regardless of the online and / or in-store experience. At step 510, user preference data such as size, color, material, type preference, etc. may be obtained. At step 515, on the front-end or user side, scanned data or graphic data may be obtained for the user, whether performed by the user or by a shopping assistant, for example, from standard photographs, 2D and / or 3D scanning, or from capture and processing using a POS self-checkout or the like. In some embodiments, a dedicated or general-purpose application on a smartphone, tablet, or other computing device may be used to enable effective user photography or scanning. In further embodiments, a dedicated or general-purpose camera or scanning device may be used by the shopping assistant, which may include portable or stationary devices, self-checkout types, or stand-alone devices. At step 520, this graphic data, along with the various input data from steps 505, 510, and 515, is used by the personalized shopping system to generate a multi-dimensional user shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. One advantage of this system is the seamless transition that allows the user to move between online and offline shopping while enjoying the benefits of personalization using a personal profile that is constantly updated in online and / or in-store scenarios.

[0079] In some embodiments, at step 525, a match of the user shopping profile to the researched or requested products is performed, optionally, against online shopping placed within a physical store. At this step, product data from the product database 500 is matched to the products requested by the user according to a particular user shopping profile, thereby highly removing products inappropriate for a particular user and enabling highly matching appropriate products according to the personal shopping profile and preferences of the particular user. This high level of screening enables, for example, a store salesperson or the user himself / herself to be presented with products that are sufficiently appropriate, optionally, products that are not appropriate and thus waste the time and resources of the shopping assistant and / or the shopper himself / herself, rather than user-selected items, with currently available products.

[0080] In step 530, product fitting data from physically present personnel or feedback from people far away can be used to assist in modifying the match - making of product data against user data. For example, feedback from in - store salespersons, or feedback from people far away connected via, for example, smartphones or computers, can be used to update the user profile. In some cases, feedback from salespersons or friends, such as which color looks good or which size looks the best, can be used by the user to update their shopping profile. In step 535, feedback can be obtained from the social network the user is connected to in order to assist in modifying the user shopping profile. In some cases, advanced graphic processing and 3D rendering can be used for the user to virtually try on the products being researched, so that the user can see themselves wearing the product according to a digital simulation that places the product on the user's shopping avatar. As described above, the user can use the shopping avatar to provide further feedback for modifying the user's shopping profile.

[0081] In step 540, products can be ordered by online users within a physical store. In step 550, personalized products can be ordered by online users within a physical store to enable the manufacture of, for example, products that are generally available but currently not in the store, or products that are specially requested based on the user's shopping avatar such that the product is a one - time customized product for the user.

[0082] In some embodiments, at step 545, a match is performed for a user in a physical store between the user shopping profile and the research or requested products. At this step, product data from the product database 500 is matched with the products requested by the user according to a specific user shopping profile, thereby highly removing products inappropriate for a specific user and enabling highly matching appropriate products according to the personal shopping profile and preferences of the specific user. This high-level screening enables, for example, a store salesperson or the user himself / herself to be presented with products that are sufficiently appropriate, and optionally, products that are not appropriate and thus waste the time and sources of the shopping assistant and / or the shopper himself / herself, rather than the user-selected items, among the currently available products.

[0083] Here, referring to FIG. 6, which is a flowchart showing an example of personalized shoe shopping, online or offline, according to some embodiments. As shown in the figure, insole information from the insole product database can be obtained at 600. Additionally, last information from the last product database can be obtained at 605. In some embodiments, the midsole data, last data, and / or shoe model data may include data regarding the shape, volume, shoe material, closure type, width, length, height, thickness, elasticity of the material, comfort fit, etc. of each product. In some examples, mesh analysis and digitization can be provided, for example, to perform mesh analysis of each 3D last in combination with 2D DXF data for addition to the last database 605. Additionally, shoe model data for discovering, purchasing, or manufacturing shoes can be obtained from the shoe model database 610. In some cases, for example, a 3D STL file can be imported for the last and a 2D DXF file can be imported for the last sole.

[0084] In step 615, the scanned data or geometric data can be obtained from the user, for example, from standard photographs, 2D and / or 3D image scanning, or capture and processing using an automatic shopping assistant device or the like. This graphic data can be processed at 620 by a personalized shopping system, either across the cloud or within the device itself, to generate a user's body profile based on the user's physical characteristics. The profile can include all or some of the following attributes: a 3D mesh including the exact geometry of the profile, a 3D mesh including the exact geometry of one or more parts of the user's body, a 2D image, a specific volume, cross-sectional measurements, characteristics calculated from one or more of the input methods including specific distances and lengths, and non-numerical attributes such as preferences. For example, while standing on a floor or another surface, on or near a reference object, etc., one or both of the user's feet can be photographed or scanned together. Further, one or both of the user's feet can be scanned by a camera that rotates around different angles of the foot, or by a person moving around the camera, thereby generating a 3D model, video, or series of images as a reference. The scanned data can typically be processed, optionally including interpolation and / or cleaning, to enable object identification and mesh generation, or other appropriate processing means. In some cases, meshing can further enable removal of extra geometry and / or correction of mesh errors, including, for example, separation, identification, and preparation of each of the two feet, removal from the scan of the floor, pants, or other extra material. Processing of the scanned data can, in some cases, enable feet alignment, which also aids in separating or individuating the two feet to provide accurate measurements of the two feet, including dimensional analysis to determine overall and region-specific lengths, widths, and heights, and cross-sectional analysis to determine areas, perimeters, and other dimensions at specific cross-sections. Processing of the scanned data can enable smoothing of the edges of the scanned feet, formation of lost volumes, construction of the soles of the feet, etc.In some cases, the complete foot volume and / or region-specific volumes can be extracted from the model as additional information. At step 625, the processed user scan data can be parameterized to extract, for example, accurate length, width, height, arch, ball, cross-section, perimeter, and volume dimensions. These parameters can be calculated from the cleaned 3D mesh using specific algorithms. For example, the calculation of the arch height of the foot of a model scanned in a standing position can be complex and can be based on the comparison of the XYZ parameters of various anatomical sites across different cross-sections at the center of the foot. The width of the foot can be calculated based on the volume of the forefoot calculated at both the 3D mesh and 2D cross-section levels. The length of the foot can be calculated from a combination of the total length and the "ball length" representing the distance between the heel of the foot and the first metatarsal bone. In addition, user-specific conditions such as pain, infection, injury, etc. can also be identified and integrated into the user's shoe shopping profile. At this step, the pronated or supinated state of the foot can also be analyzed. Additionally, for the fitting of insoles or other orthotics, the arch of the foot can be identified and measured.

[0085] At step 630, historical data for the user can be obtained, for example, based on previous user purchases at a store or chain store, regardless of the online and / or in-store experience. At step 635, user preference data such as size, color, material, type preference, etc. can be obtained. At step 640, a multi-dimensional user shoe shopping profile (hereinafter referred to as the user shopping avatar) can be created by processing the various input data from steps 615, 620, 625, 630, and 635, thereby generating a user shoe shopping avatar or profile that includes the user's physical characteristics as well as user behavior and user preference data. In some embodiments, the user shoe shopping profile includes both of the user's feet, which are typically different, and parameters are individually determined for them, thereby benefiting from individual profiles for the left and right feet.

[0086] In step 645, a match is performed for the shoes being considered, researched or requested in the user shoe shopping profile. In this step, product data including dimensions from product databases 600, 605 and 610 is matched to the products being researched by the user according to a particular user shopping profile, thereby enabling a sophisticated removal of products not suitable for a particular user, as well as a sophisticated matching of suitable products according to the personal shopping profile and preferences of a particular user. The matching step may be complemented by providing recommendations to the user based on the match-making process for the shoes of the profile described above. This step may use foot models and digital profiles of shoes directly, as well as / or parameterized numerical models.

[0087] In step 650, product fitting data from physical attendants or feedback from people far away can be used to assist in modifying the match-making of user data to product data. For example, feedback from in-store salespersons, or feedback from people far away connected via, for example, smartphones or computers can be used to update the user profile. In some cases, feedback from salespersons or friends, such as which color looks good or which size looks the best, can be used by the user to update their shopping profile. In some cases, advanced graphic processing and 3D rendering can be used for the user to try on the product being researched, so that the user can virtually see themselves wearing the product according to a digital simulation that places the product on the user's shopping avatar. As described above, the user can use the shopping avatar to provide further feedback to modify the user's shopping profile. In step 655, feedback can be obtained from the social network the user is connected to in order to assist in modifying the user shopping profile.

[0088] In step 660, personalized shoes can be ordered by the user in a physical store or an online store. Further, the personalized product can be ordered from a manufacturer that can manufacture the product based on the user's requirements so that the product becomes a one-time customized product for the user. The customized footwear according to the present invention can be customized and / or personalized, for example, in one or more of the following ways: shape (e.g., size, length, geometry, volume), design (e.g., color, pattern, print, material) or any other specification or combination of the foregoing.

[0089] Referring now to FIG. 7, which is a flow diagram showing examples of personalized eyewear shopping and manufacturing, either online or offline, according to some embodiments. This embodiment refers to both sunglasses and any type of optical glasses. As shown in the figure, eyewear model information from an eyewear product database is at 700 and can be obtained from an eyewear model or product database of various file types or data structures. According to some embodiments, for example, by generating model parameters for a plurality of input types, a parameter model is prepared, resolution adjustment is processed to enable rigging and skinning, and by integrating the range of motion animations and shape-keys for correction, etc., eyewear frame acquisition can be enabled.

[0090] At step 705, for example, historical data for the user, including facial anatomical landmarks and measured distances such as, for example, an intermediate general distance, the position of each eye, the cheeks, the temples, and various points on the ears and nose, can be used. General parameters for each face can also optionally include volume, ratio, and standard shape, and can be loaded and obtained, for example, based on previous user inspections and / or purchases at a store or chain store, regardless of the online and / or in-store experience. At step 710, user preference data such as size, color, material, type preference, usage requirements, etc., can be obtained. This data can be obtained directly or indirectly, for example, using purchase information, questionnaires, forms, and / or any other data acquisition method.

[0091] In step 715, the scanned data or graphic data can be obtained from the user, for example, from standard photographs, 2D and / or 3D image scanning, or capture and processing using an automatic shopping assistant. This includes alternative 2D methods that may or may not use any type of 3D scanning technology or reference objects for sizing as described in the previous section. Typically, scan data of the user's head and face can be obtained at this stage. In some embodiments, the preparation and analysis of the head and face model can include model preparation such as smoothing and cleaning, reconstruction of the mesh with improved topology and / or optimized detail / weight ratio compression for real-time optimal display, and / or orientation and placement. Additionally, for example, facial feature recognition can be used to generate a facial model rendered from multiple angles (e.g., 3 to 15 angles, to assist in determining depth, color, etc.). Moreover, computer vision and / or machine learning algorithms can be applied to identify the eyes, nose, nasal bridge, cheeks, ears, etc. The processing of the scanned data can further include projection of 2D landmarks onto a 3D model and verification of anatomical landmarks. In the absence of landmarks, an evaluation based on statistical or empirical results can be applied for the replacement of these landmarks.

[0092] In step 720, the processed user scan data can be parameterized to extract, for example, the exact face length, width, height, ratio, nose width and volume, ear size, ear height, ear position, skin color, and / or other relevant facial features and dimensions. This data, together with the user history data from 705 and the user preference data from 715, is processed by the personalization shopping system at 725 to generate a user's face profile and other physical characteristics, as well as a user's glasses shopping profile based on the user behavior and user preference data. In some embodiments, the user glasses shopping profile includes both of the user's eyes, which are typically different, thereby benefiting from individual profiles for the left and right eyes. In some embodiments, the optical prescription will be collected using a photograph, an electronic form, or an inspection method embedded within the application, or in some cases from an external prescription file. According to some embodiments, the system may recommend specific models based on the face shape, previous purchases and history, and optionally, based on a comparison with similar avatars (optionally anonymized).

[0093] In step 730, a match-making for the user glasses shopping profile against the researched or requested products is performed. In this step, the product data from the product database 700 is matched with the products being researched by the user according to a specific user shopping profile, thereby highly removing the products inappropriate for a specific user and enabling the highly matching of the appropriate products according to the specific user's personal shopping profile and preferences.

[0094] In step 745, each frame is tried on the user's face using frame customization, which may include, for example: making key measurements regarding the face of the subject or user; applying an iterative comparison algorithm to fit the frame to the face; digitally positioning the frame on the subject's face according to the prepared user eyewear shopping profile or avatar, properly orienting the frame with respect to the face, magnifying and reducing the bridge size, width and position, and adjusting the folding of the arms, arm length and pentoscopic tilt or angle, etc.

[0095] In step 735, product fitting data from physical attendants or feedback from people far away can be used to assist in modifying the matchmaking of user data to product data. For example, feedback from in-store salespersons or optometrists, or feedback from people far away connected via, for example, smartphones or computers, can be used to update the user profile. In some cases, feedback from salespersons or friends, such as which color looks good or which size, style, type looks best, can be used by the user to update their shopping profile.

[0096] In some embodiments, advanced graphics processing may be used for the user to virtually try on the product being researched, whereby the user can see themselves wearing the glasses according to a digital simulation that places the glasses on top of the face of the user shopping avatar. Virtual try-on may, in some embodiments, include features such as physically simulating placing the glasses correctly or in an appropriate position and being able to slide them along the nose. Additionally, try-on may include overlaying a photo or 3D model on the face model / or a series of photos or any combination of the foregoing. Animating effects may be included to highlight different attributes of the glasses, such as a customization animation in the case of a custom frame, or other animations such as fly-in / fly-out animations for swapping between glasses. As described above, the user may use the shopping avatar to provide further feedback to modify the user's shopping profile. In some embodiments, the user can see the customized frame on a digital version of their face to provide meaningful visual feedback. For example, the user's face may be displayed in a 3D viewer and the appearance enhanced to provide one or more 3D viewing operations (e.g., zooming, rotation), and animating effects that complement the user experience, such as a face breathing, smiling, blinking or other animating or static visual effects. Further, the user may thus be provided with customization options including selecting any frame from the collection, customizing the frame and lens colors, customizing the auto-recommended fit, personalizing files (e.g., text, prescription, etc.), and enabling side-by-side comparison of different frames. At step 740, feedback may be obtained from the social network the user is connected to in order to assist in modifying the user shopping profile. Needless to say, other combinations of steps may be used to process the input data.

[0097] Step 750 refers to an embodiment in which the system can manufacture custom eyewear based on automated or semi-automated parameter design that adapts the frame to the user. In step 750, the relevant eyewear manufacturing printing and cutting files can be prepared by the system if necessary. In some embodiments, 3D printing files in standard form, such as STL or OBJ, and 2D lens cutting files such as DXF can be prepared. In some embodiments, the system creates two or more pairs of models for each frame design. This allows, for example, maintaining a high-resolution model for printing file preparation that includes high-resolution features and details such as hinges, grooves, angles, temples, etc., while using a lightweight model for visualization purposes at the front end of the application. The eyewear customization of the printing models described herein can be automatic and / or manual. Further, file preparation for 3D printing can include automatically fixing printability issues, generating standards, duplicates, holes, and non-manifold geometries. In some embodiments, the system can create custom tags, labels, or other product identification or recognition technologies or devices on the eyewear or eyewear, including text, QR, or barcodes that enable traceability throughout the manufacturing and sales process.

[0098] In step 755, personalized eyewear can be ordered by the user, whether in a physical store or an online store. Further, the personalized product can be ordered from the store upon request or from a manufacturer that can manufacture the product based on the user's request such that the product is a one-time customized product for the user.

[0099] According to one embodiment, the system and process are described for automated personalized product ordering using digital mirrors or personalized display protocols. This embodiment integrates virtual reality and / or augmented reality for operating, displaying, and / or trying on a specified fixture, such as existing or designed glasses, on a screen, table, smartphone, communication device, etc., to enable visual display of a custom or non-custom frame on a client's face.

[0100] According to one embodiment, a file format is provided that is adapted to enable ordering of personalized products. This file format incorporates all relevant information, including physical characteristics and personal preferences, to represent the user and assist the user in performing personalized or non-personalized shopping for clothing, eyewear, or other body-related products. This avatar standard format can be plugged into online or physical, substantially any shopping platform to enable store customization to fit the customer's physical and aesthetic needs and preferences.

[0101] Referring now to FIGS. 8A-8G, which show different displays of a POS device, a start pad, or a vending machine, according to some embodiments. In some embodiments, the shopping assistant device generates a user avatar based on one or more of 3D scanning, image acquisition using one or more cameras, measurement of user profiles using pressure plates, and the like. In some embodiments, a positioning pad or a standing pad that functions as a reference for the position where the user stands to perform a body scan may be used. In some cases, the standing platform includes lighting at the bottom of the standing pad to minimize or eliminate shadows and cause a black / white peripheral effect to support the acquisition of accurate measurements of any color from a camera or scanner on the device. In some embodiments, there may be proximity sensors for each foot - measuring when the foot is in place or near it and telling the user to move if necessary. Further, the device may include a distance or proximity sensor / robot that recognizes a user approaching to attract the user to the device, in which case the device may be automatically started when the user enters a selected geographical zone. In further embodiments, sensors may be used to measure the height of the user's instep - for example, to establish the volume of the instep. In still further embodiments, specialized sensors, such as sensors for measuring diabetic ulcers and the like, may be used. Such sensors, such as cameras, 3D sensors, panoramic cameras, and / or multi-eye cameras, may be used.

[0102] FIG. 8A is an exploded view of an example of components of a POS device or a vending machine in some embodiments. As shown in the figure, the vending machine 800 includes a standing base 805, a light source for illuminating the standing base optionally, proximity sensor(s) 816 for identifying the positioning of the feet to be scanned by the vending machine 800, a standing pad 815 having a space for standing on both feet, and a pad layer 820 on which the feet are placed. In some embodiments, the pad layer 820 may integrate a pressure detection mechanism such as a flatbed scanner, a pin plate, a pressure plate, a touch screen type surface with capacitors and / or pressure sensor elements, etc. to determine the arch dimension of the foot. In some embodiments, the arch dimension measuring element(s) may be used to determine the need for and / or the dimension of the insole for the user.

[0103] In some embodiments, the arch dimension measuring element(s) may be used to determine the need for and / or the dimension of the instep profile of the user's foot.

[0104] In some embodiments, the arch dimension measuring element(s) may be used to determine the need for and / or the dimension of the ball profile of the user's (foot).

[0105] In other embodiments, one or more lasers or other illumination mechanisms may be used to determine the dimensions of the arch, ball, or instep of the foot. In one example, a laser from above the foot is used to identify, for example, the size of a "hidden" area not visible by the laser, and by using that "hidden" space to determine parameters such as height at different points on the foot, the height or the size of the arch and the bridge of the foot may also be indicated. In a further example, a diffraction grating, a prism, or other filters using multiple lines are used to enable the identification of the highest point of the foot.

[0106] In yet further embodiments, one or more lights, optionally of different colors, may be used, separately and / or in combination, with image processing to neutralize the color of the socks and / or the feet, to assist in identifying spaces that are not the feet. In other embodiments, for example, background removal techniques may be used to identify one or more along with the arches of the feet.

[0107] Furthermore, the vending machine 800 may include a main body 822 that includes a computer holding stand 825 and a camera holding element 826 for holding one or more camera elements 835, a panel element 830, a further panel or cover element 845, a computing screen, preferably a touch screen PC or tablet 840, and optionally has a location for setting up a proximity sensor 850 for identifying, for example, a user in proximity to the computing device.

[0108] FIG. 8B is a front view of an example of a POS device or a vending machine.

[0109] FIG. 8C is an isometric view of an example of a POS device or a vending machine.

[0110] FIG. 8D is an isometric front view of an example of a POS device or a vending machine.

[0111] FIG. 8E is an isometric rear view of an example of a POS device or a vending machine.

[0112] FIG. 8F is a side view of an example of a POS device or a vending machine.

[0113] FIG. 8G is a top view of an example of a POS device or a vending machine.

[0114] FIG. 8H is a diagram of an example of one or more sensors in a POS device or a vending machine. As shown in the figure, in the vending machine, the plurality of sensors may constitute, for example, one or more proximity sensors, and the LED flatbed may be configured to provide a light source from below the standing area.

[0115] Referring now to FIG. 9, a schematic diagram of a shopping assistant system 900 is shown that includes an automated shopping assistant device 906, which integrates computing components adapted to execute a shopping assistant application or software program 915. The automated shopping assistant device 906 is adapted to connect to a communication network, such as a communication cloud 920, to access and / or transmit a user shopping profile 925 to the cloud when it is located on the cloud. Further, remote user mobile devices, such as smartphones, tablets, or other mobile communication devices 905, 910 that support a camera, are adapted to execute a shopping assistant application or software program 916 according to some embodiments. Devices 905 and 910 typically include one or more cameras 910 and are capable of capturing information regarding a person's one or both feet, for example, using standard image / scans, video, a series of images, or advanced detection components such as stereo illumination, time-of-flight, or others in IR / NIR or visible light conditions, optionally simultaneously. Devices 905 / 910 typically include a gyroscope to provide camera orientation data to the device camera, for example, to enable an image to be acquired only when the camera is substantially flat. In some embodiments, internal detection components and / or additional sensors may be attached to communication devices 905, 910 to supply supplementary data to the system. Such sensors, for example, may assist in improving the accuracy of measurements, capture data with real-time feedback, and / or assist the user while providing input to a computing engine. Remote devices 905 and 910 may communicate with the communication cloud 920 and, specifically, may be connected to the digital shopping avatar or profile 925 of the device user.In some embodiments, a user may generate a shopping profile for one or more users using mobile devices 905, 910 in addition to or instead of the automatic shopping assistant device 906.

[0116] Referring now to FIG. 10, which shows a schematic diagram of a shopping assistant system 1000 and the workflow between components. As shown in the figure, a start pad or shopping assistant device 1005 can scan a user to generate a shopping profile for the user. The generated profile is sent to the user's mobile device 1010 and can then be used to scan products, such as shoes 1015, using product tags, such as QR code 1020, which represent the selected product. Optionally. Further, in some embodiments, a user may build a shopping profile using a mobile application 1025. In some embodiments, the user shopping profile can be used at 1030 to enhance in-store shopping. In some embodiments, the user shopping profile can be used at 1040 to enhance online shopping.

[0117] Refer to FIG. 11A, which is a flowchart showing an example of personalized footwear shopping in a store where an in-store or POS automatic shopping assistant device is used in combination with a mobile computing device application. As shown in the figure, at step 1100, the user can be identified by the proximity sensor of the shopping assistant device. At step 1105, the user can be requested to stand on the marked pad, and optionally, interactive guidance can be provided to ensure that the user stands in the correct position to enable an accurate scan. At step 1110, the device scans the body or body part / element, and at step 1115, generates a shopping profile based on that scan. At step 1120, the device can present a shopping avatar to the user in a graphic format. At step 1125, the device can send the shopping avatar to the user's mobile device. At step 1130, the user can open a shopping application that can be used to assist with shopping and / or research. At step 1135, the user can use the avatar on their mobile device by scanning a selected product, for example, using a product identifier or recognition device or technology such as a coded tag, QR code (registered trademark), or barcode, to shop or research. Generally, the scanned product can be processed in a way related to the user avatar to determine, for example, whether the product fits the avatar or is suitable for the user's profile or preferences. In some cases, the application can provide a graphic simulation of the selected product on the avatar. At step 1140, the application connects to a communication cloud or other database to match the selected product(s) with product data to assist in determining, for example, purchase options, inventory status, product quality, product features, reviews, sizes, etc. In some cases, at step 1145, the application can present recommendations, purchase data, etc. to the user.In a further step, the application may enhance the in-store shopping experience by providing, for example, shopping advice, options, shortcuts, access to additional databases, and the like.

[0118] Refer to FIG. 11B, which is a flowchart showing an example of personalized footwear shopping in a store where an in-store or POS automated shopping assistant device is used in combination with a mobile computing device application. As shown in the figure, at step 1100, a new or known user can enter onto the shopping assistant device, for example, via a biometric identifier, an entry screen, etc. At step 1105, the user may be requested to stand on a marked pad, and optionally, interactive guidance is provided to ensure that the user stands in the correct position to enable an accurate scan. At step 1110, the device scans the body or body part / element, and at step 1115, the scan data is processed to generate a shopping profile based on the scan. At step 1120, the device may present the shopping profile to the user in a graphic or other format as a simulation, a shopping avatar, or other virtual assistant. At step 1125, the device may transmit the shopping avatar to the user's mobile device in a format or configuration that can be used by the software, code, or application(s) of that device. At step 1130, the user can open a shopping application or other program that can operate the shopping avatar with the shopping assistant application. At step 1135, the user can use the avatar on their mobile device by scanning a selected product, for example, using a product identifier such as a coded tag, QR code, or barcode, for shopping or research. Generally, the scanned product can be processed in a way that incorporates the generated shopping avatar to determine, for example, whether the product fits the avatar or conforms to the user's profile or preferences. In some cases, the application may provide a graphic simulation of the selected product on the avatar.In step 1140, the application can connect to a communication cloud or other database to match the selected product(s) with advanced product data to assist in determining, for example, purchase options, inventory status, product quality, product features, reviews, sizes, etc. In some cases, in step 1145, the application can present recommendations, purchase data, purchase options, reviews, news, etc. to the user to assist in enhancing the online store shopping experience. In some cases, in step 1150, the application can present recommendations, purchase data, purchase options, reviews, news, etc. to the user to assist in enhancing the in-store shopping experience by providing, for example, shopping advice, options, shortcuts, access to additional databases, etc.

[0119] In some embodiments, the user shopping experience can be performed for additional users connected to the user of the shopping assistant. In such cases, the user application includes the shopping profiles of multiple users, thereby enabling the user of the mobile device to perform shopping for multiple users according to that user's shopping profile.

[0120] According to some embodiments, the mobile and / or user avatar can be shared with other users. For example, a user can obtain access to, or manage, multiple user profiles with the user's approval, for example, within a wallet or holder of the avatar or profile. In such cases, the managing user can shop on behalf of other users. For example, a parent can hold the profiles of all of their family members and enable the parent to easily shop online and / or offline for all of the associated family members.

[0121] According to some embodiments, additional personalization, such as an icon or a photo, can be provided for each user mobile shopping avatar or user shopping avatar. Such personalization can be particularly useful for a managing user handling multiple users or mobile shopping avatars. This information, associated with any identifier, can be stored on a cloud avatar database and associated with the user within any platform they use. For example, if a member of a family scans and saves one or more of the profiles of their family members, these can be shared with another member(s) of the family who can currently load them on the in-store system or e-commerce website being used and later use this personalized information. In the case of a website, the output can be personalized according to this personalized data and can include, even a photo or avatar or 3D model of another user, next to the information being provided, with the recommendations being personal and reassuring the user's trust that they are based on his or her profile, or other profiles legally used by the user.

[0122] According to some embodiments, an online store may include a user shopping virtual assistant that can appear on substantially any web page (regardless of whether optimized for mobile phones, desktops, notebooks, tablets, wearables, etc.) using a profile plugin or other digital object to provide recommendations, guidelines, or other assistance to the user when appropriate. For example, the virtual shopping assistant may display information regarding different fits for the user profile being used or otherwise assist the user. For example, a user with a foot size of European 41 may be notified that the size corresponding to the user's foot profile in Nike shoes is European 42 or US size 10.5 while shopping or browsing within the Nike online store. Additionally, if the user profile includes preference data such as preferred colors and fits, the shopping virtual assistant may also provide suggestions or guidelines based on the user's preferences. For example, in a Nike shoe store, the shopping assistant may suggest to the user to look for options such as running shoes in either blue or green in a US size 10.5.

[0123] In some embodiments, the virtual assistant may direct the user directly to pages or a plurality of pages that match the user's shopping profile data and preferences. In one embodiment, the profile may move the website to sections that may be of interest or appropriate for a particular user while avoiding irrelevant pages. In another embodiment, the system may use the personalization information, alone or integrated with additional users, to reorganize the website and create a personalized version of the website that may be of most interest to him / her and represent what may best suit him or her.

[0124] According to some embodiments, the virtual shopping assistant may enable rendering of a 3D view of the products being displayed and, optionally, of personalized products. For example, custom shoes being displayed according to a user's shopping profile may be rendered in 3D from all sides and angles to assist the user in viewing the product in multiple dimensions.

[0125] According to some embodiments, a virtual fitting module may be provided to enable a shopping avatar to wear the product(s) being displayed.

[0126] According to some embodiments, the user shopping avatar may be a one-time avatar for a store. In another embodiment, the user shopping avatar may be applicable to a chain store. In other embodiments, the user shopping avatar may be applicable to various brands or stores, for example, all owned by the same parent corporate entity. In other embodiments, the user shopping avatar may be applicable to any or all stores by connection to a universal user profile in the cloud.

[0127] Reference is now made to FIGS. 12A - 12B, which are examples of screenshots showing an interactive screen of a shopping assistant screen for guiding a user to place a foot on a marked pad, according to some embodiments.

[0128] Reference is now made to FIGS. 13A - 13B, which are examples of screenshots showing an interactive guide on a shopping assistant screen or a mobile screen for assisting a user in defining their profile and contact information, according to some embodiments.

[0129] Reference is now made to FIGS. 14A - 14B, which are examples of screenshots on a shopping assistant screen or a mobile screen showing a simulated rendering of a scanned pair of feet and calves, according to some embodiments.

[0130] Refer to FIGS. 15A - 15B, which are examples of screenshots of a shopping assistant screen or a mobile screen for assisting a user in entering behavior - related information, which can be used to provide better user - related output.

[0131] In some embodiments, the shopping assistant device can assist a chain store in generating loyalty online and / or offline by enabling an automatic shopping assistant to perform up - sales and cross - sales, etc. in multiple stores.

[0132] In one embodiment of the present invention, the user shopping experience for footwear can be substantially enhanced by applying the following steps: measuring the user's physical profile on device 185 to provide the user's standard size and modifications for different shoes / brands; obtaining the customer's ID or shopping profile for the customer's mobile computing or communication device; obtaining the customer's ID or shopping profile from device 185 to the communication cloud.

[0133] In the first user scenario of the present invention, the user does not have a previous user profile, and the shopping assistance process can be implemented as follows: The user typically removes their shoes and stands on a start pad or a shopping assistant device at the entrance of the store or in the shopping area. In the current example, a footwear application is described. The device then measures / scans the user's body area, such as both feet, and then the device or a cloud network can process the user data to generate a user shopping avatar. Once generated, the avatar is sent to the user's mobile device running the shopping assistant application, for example, using email, beacon, SMS, QR code, IR beam, etc. The user can then scan the desired shoes, whereupon the device is configured to match the desired shoes to the avatar and then have them tried on to provide the best fit. The device can also provide user-related product information, such as availability, color, size, related shoes, rankings, etc.

[0134] In the second user case of the present invention, the user has multiple devices, and the shopping assistance process can be executed as follows: The user usually takes off their shoes at the entrance of the store or in the shopping area and stands on the start pad or the shopping assistant device. In the current example, a footwear application is described. The device then measures / scans the user's body area, such as both feet, and then the device or the cloud network can process the user data to generate a user shopping avatar. Once generated, the avatar is sent to the user's mobile device running the shopping assistant application, for example, using email, beacon, SMS, QR code, IR beam, etc. The user can then scan the desired shoes, whereupon the device is configured to match the desired shoes to the avatar and then allow a trial fit to provide the best fit. In the current embodiment, the device can provide the user with expert or consulting information, thereby functioning at least partially as a salesperson. The device can also provide the user with related product information, such as availability, color, size, related shoes, rankings, etc., and options to measure things like exercise / style / weight.

[0135] In the third user case of the present invention, an enhanced shopping experience is delivered, whereby the automated shopping assistance process can incorporate local shoe scanning. In some cases, the user data or avatar can be used to sort through appropriate reviews / comments from the online world + social feedback + rankings, sales information / history, recommendations, upselling, cross-selling, etc.

[0136] The foregoing description of the embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications, variations, substitutions, alterations, and equivalents will be apparent to those skilled in the art in light of the foregoing teachings. Accordingly, it is understood that the appended claims are intended to cover all such modifications and that changes are within the scope of the true spirit of the invention.

Claims

1. A shopping assistant device; a mobile device application; 1. A personalized shopping assistant system, comprising: The shopping assistant device is configured to capture a body part of a user and generate a user profile based on capturing the body part of the user, wherein capturing the body part of the user includes: acquiring at least one image of the body part of the user; generating a 3D mesh model of the body part based on the at least one image; the 3D mesh model includes user measurement data of the body part; the mobile device application configured to apply the user profile in a database using the mobile device to match one or more products to the user profile based on the 3D mesh model; the shopping assistant device is configured to transmit the user profile to the mobile device of the user to provide a fit of at least one selected product on the user profile selected from one or more products matched to the user profile, and providing the fit of the at least one selected product on the user profile includes providing a color heat map of the at least one selected product fitted on the user profile to indicate at least one of tightness, pulling, or chafing on the body part of the user; Your personalized shopping assistant system.

2. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device is further configured to provide a scan mat including markings to guide the user to stand appropriately to enable accurate scanning of the body part of the user, and the at least one image is captured when the body part is within the markings.

3. 2. The personalized shopping assistant system of claim 1, wherein generating the user profile includes generating a user shopping avatar based on capturing the body parts of the user, and the shopping assistant device is configured to display a digital simulation of the fit of at least one selected product on the user shopping avatar.

4. 4. The personalized shopping assistant system of claim 3, wherein the shopping assistant device is configured to provide a machine-based digital representation configured to interactively provide guidance to the user for selection and customization of the at least one selected product.

5. 4. The personalized shopping assistant system of claim 3, wherein the digital simulation of the fit comprises a live stream of animated video showing the user shopping avatar wearing the at least one selected product.

6. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device includes one or more sensors configured to capture a depth of the body part of the user, and the shopping assistant device is further configured to perform a dimensional analysis to determine one or more of a length, a width, a height, or a cross-sectional analysis of the body part.

7. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant device is further configured to process the scanned image data to extract one or more of length, width, height, arch, bowl, cross-section, circumference and / or volumetric dimensions associated with the body part of the user.

8. 2. The personalized shopping assistant system of claim 1, wherein generating the 3D mesh model includes generating one or more user-specific conditions associated with the body part, the user-specific conditions including one or more of a pain, an infection, a damaged area, an injury condition, or a prosthetic measurement.

9. 2. The personalized shopping assistant system of claim 1, wherein the shopping assistant apparatus is further configured to receive feedback regarding the fit of the at least one selected product on the user via one or more remote computing devices, each remote computing device being associated with a different user, and to provide one or more matched products modified based on the feedback.

10. The personalized shopping assistant system of claim 1 , wherein the shopping assistant device is further configured to generate a customized product order based on user product selections and the 3D mesh model.

11. 1. A method of matching a user with a footwear article, comprising: the shopping assistant device initiating capture of a body part of the user using one or more integrated imaging sensors; acquiring, by the shopping assistant device, at least one image of the body part using the one or more integrated imaging sensors; generating, by the shopping assistant device, a 3D mesh model of the body part based on the at least one image, the 3D mesh model including user measurement data of the body part; generating a user profile for the user, the shopping assistant device generating a user profile for the user, the user profile including the 3D mesh model of the body part; the shopping assistant device transmitting the user profile to a user mobile device; providing the user with automated product matching for one or more products based on matching the one or more products to the user profile based on the 3D mesh model, the shopping assistant device providing at least one selected product fit on the user profile selected from one or more products matched to the user profile; and providing, by the shopping assistant device, a machine-based digital representation configured to interactively provide user guidance for selection and customization of the at least one selected product. method.

12. 12. The method of claim 11, further comprising providing a scanning mat including markings to guide the user to stand appropriately to enable accurate scanning of the body part of the user, and wherein the at least one image is captured when the body part is within the markings.

13. 12. The method of claim 11, wherein generating the user profile includes generating a user shopping avatar based on capturing the body parts of the user, and the shopping assistant device is further configured to display a digital simulation of a fit of the at least one selected product on the user shopping avatar.

14. The method of claim 13 , wherein the digital simulation of the fit comprises a live stream of animated video showing the user shopping avatar wearing the at least one selected product.

15. 14. The method of claim 13, further comprising, in response to a user selection, the shopping assistant device transmitting a personalized product order to a production facility to enable production of a product customized based on the user shopping avatar.

16. 12. The method of claim 11, further comprising: capturing a depth of the body part of the user with one or more sensors; and performing a dimensional analysis to determine one or more of a length, width, height, or cross-sectional analysis of the body part.

17. 12. The method of claim 11, further comprising processing the scanned image data to extract one or more of length, width, height, arch, ball, cross-section, circumference and / or volume dimensions associated with the body part of the user.

18. 12. The method of claim 11, wherein generating the 3D mesh model comprises generating one or more user-specific conditions associated with the body part, the user-specific conditions comprising one or more of a pain, an infection, a damaged area, an injury condition, or a prosthetic measurement.

19. 12. The method of claim 11, further comprising receiving feedback regarding the fit of the at least one selected product on the user via one or more remote computing devices, each remote computing device being associated with another user, and providing one or more matched products modified based on the feedback.

20. 12. The method of claim 11, further comprising: generating a customized product order based on user product selections and the 3D mesh model; and connecting a production system to facilitate manufacturing of the customized product represented on the 3D printer file.

Images