Camera platform and object inventory control

Abstract

Camera platforms and object inventory control techniques are described. In implementations, a live feed of digital images is output by a computing device in a user interface. A user selection is received by interacting with the user interface of at least one of the digital images. Machine learning is used to identify an object included within the at least one digital image. Metadata related to the identified object is then obtained. Augmented reality digital content is generated based at least in part on the obtained metadata. The augmented reality digital content is displayed as part of the live feed of digital images associated with the object.

Description

[0001] Related applications

[0002] This application claims priority to U.S. Application 15 / 858,463, filed December 29, 2017, entitled "Camera Platform and Object Inventory Control," and U.S. Provisional Patent Application No. 62 / 558,836, filed September 14, 2017, entitled "Camera Platform and Object Inventory Control," pursuant to 35 U.S. SC §119(e). The entire disclosure of both applications is incorporated herein by reference. Background Technology

[0003] Mobile devices have become an indispensable part of users' daily lives. For example, mobile phones can be used to read emails, participate in social media, capture digital images, communicate via instant messaging, and so on. Similarly, wearable devices such as smartwatches continue to expand this interaction. As a result, users can access a wide variety of devices in various use cases.

[0004] However, devices as mobile devices can introduce challenges and complexities in supporting user interaction with them. For example, mobile phones or smartwatches may have limited capabilities in supporting text input, navigation between documents, and so on. Consequently, user interaction with these devices may be restricted, leading to computational inefficiency. Summary of the Invention

[0005] A camera platform and object inventory control technology are described. In this implementation, a live feed of digital images is output by a computing device to a user interface. User selections are received through interaction with the user interface of at least one of the digital images. Machine learning is used to identify objects included within the at least one digital image. Metadata related to the identified objects is then obtained. Augmented reality digital content is generated, at least in part, based on the obtained metadata. The augmented reality digital content is displayed as part of a live feed of digital images associated with the objects.

[0006] This invention summary presents a series of concepts in a simplified form, which are further described below in the "Detailed Description" section. Thus, this summary is not intended to identify essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter. Attached Figure Description

[0007] The accompanying drawings are described in detail. The entities shown in the drawings may refer to one or more entities, and therefore, in the discussion, reference may be made interchangeably to a single or multiple forms of entities.

[0008] Figure 1 This is a diagram illustrating an environment that can be operated using the camera platform technology described in this article in the example implementation.

[0009] Figure 2 A more detailed illustration of the example implementation is provided. Figure 1 The system operates the camera platform manager module.

[0010] Figure 3 Examples of user-camera platform manager module interactions are depicted, as shown in the first, second, and third phases, for collecting metadata related to objects identified in digital images.

[0011] Figure 4 This section describes an example of the identified objects and their corresponding metadata, as well as an example of the storage results of the collection of user-defined objects.

[0012] Figure 5 Describes being configured to use Figure 1 The camera platform manager module supports a user interface implementation for buying and selling identified objects.

[0013] Figure 6 It also describes how the output of the camera platform manager module is used to provide options for buying or selling objects identified from digital images.

[0014] Figure 7 The system in the example implementation is depicted, showing a more detailed adoption of the user profile. Figure 1 Operations of the camera platform manager module.

[0015] Figure 8 An example implementation of user profile generation based at least in part on machine learning is described.

[0016] Figure 9 Depicting based on Figure 8 The generated user profiles provide a recommended example system.

[0017] Figure 10 An example system is shown, including various components of an example device that can be implemented as a reference. Figures 1 to 9 Any type of computing device described and / or utilized to implement embodiments of the techniques described herein. Detailed Implementation

[0018] In the following discussion, example environments in which the techniques described herein may be employed are first described. Example processes and systems are also described and illustrated as boxes that can be executed in the exemplary environment as well as in other environments. Therefore, the execution of the example processes is not limited to the example environments and systems, and the example environments and systems are not limited to the execution of the example processes.

[0019] Example Environment

[0020] Figure 1 This is an illustration of a digital media environment 100 operable in an example implementation of the digital image capture session technology described herein. The illustrated environment 100 includes a computing device 102 communicatively coupled to a service provider system 104 via a network 106. The computing devices implementing the computing device 102 and the service provider system 104 can be configured in various ways.

[0021] For example, a computing device can be configured as a desktop computer, a laptop computer, a mobile device (e.g., a handheld device such as a tablet or mobile phone), or configured to be worn (e.g., glasses as shown for computing device 102), etc. Therefore, the range of computing devices can be from full-resource devices with large amounts of memory and processor resources (e.g., personal computers, game consoles) to low-resource devices with limited memory and / or processing resources (e.g., mobile devices). Furthermore, although a single computing device is shown, a computing device can represent multiple different devices, such as multiple servers used by an enterprise to perform "cloud" operations for service provider system 104, such as… Figure 10 As described in [the text].

[0022] Computing device 102 is shown as being worn by user 108 in a physical environment such as living room 110. Computing device 102 includes a digital camera 112 configured to capture digital images 114 of the external physical environment (e.g., living room 106) such as by using a charge-coupled device (CCD) sensor. The captured digital images 114 can then be stored as pixels in a computer-readable storage medium and / or rendered for display by a display device such as LCD, OLED, LED, etc.

[0023] The computing device 102 also includes a camera platform manager module 116, which is configured (e.g., by using a processing system and a computer-readable storage medium) to implement and execute a camera platform 118 that can serve as the basis for various functions. For example, the camera platform 118 can implement a "live view" formed by digital images 114 captured from the physical environment of the computing device 102. These digital images 114 can then be used as the basis for supporting other functions.

[0024] An example of this functionality is shown as the Object Inventory Manager module 120. The Object Inventory Manager module 120 represents a function for managing the inventory of objects. This can include objects owned by user 108 and / or objects that user 108 expects to purchase, for example. This can be implemented by the Object Inventory Manager module 120 in various ways using the camera platform 118.

[0025] In this first example, the object inventory manager module 120 is configured to collect digital images 114. This could include digital images 114 of physical objects in the living room 110 in this example, or digital images captured from physical photographs, such as those from magazines, television screens, or other display devices. Digital images 114 could also be captured from user interfaces output by the computing device 102, such as screenshots from a frame buffer.

[0026] The object inventory manager module 120 includes object recognition functionality to identify objects included within the digital image 114, for example, via machine learning. Thus, the object inventory manager module 120 can collect data related to this recognition. For example, data describing the identified objects can be transmitted via network 106 to the service provider system 104. The service provider system 104 includes a service manager module 122 configured to obtain object-related data from storage device 124 (e.g., through search). This data can then be transmitted back via network 106 to computing device 102 for use by the object inventory manager module 120.

[0027] For example, the object inventory manager module 120 can generate augmented reality digital content 126 (shown as being stored in storage device 128) to be output to the user interface of computing device 102 as part of a "live feed" of digital images taken from a physical environment, such as a living room. For example, the AR digital content 126 can describe the object's characteristics, the object's brand name, the price at which the object can be sold or purchased (e.g., via an online auction), etc. This AR digital content 126 is then displayed by the object inventory manager module 120 near the object. In this way, the camera platform supports the user 108 to "look around" the living room 110 and additional object information, as well as gain a deeper understanding of the characteristics of objects included within the physical environment. The following discussion... Figures 2 to 6 Further discussion describing this example.

[0028] In another example, the object inventory manager module 120 utilizes the camera platform 118 to make recommendations for the user. For example, the digital image 114 can also be processed by the object inventory manager module using object recognition, such as that implemented using machine learning. In this example, the digital image is used to generate a profile (e.g., a user profile) based on features learned from the digital image 114. This profile is then used as the basis for forming recommendations (e.g., through machine learning), such as configuring digital marketing content with product suggestions based on these features.

[0029] For example, a profile may include a digital image taken from user 108. Thus, the object inventory manager module 120 can determine the user's possible size (e.g., dimensions), body type (e.g., how the user wears clothing, such as tight, loose, or how the clothing "hangs" on the user in other ways), style (e.g., professional, hippie, grunge, medieval), etc. Digital images can also be collected from sources other than the user but which the user desires, such as other people in person, physical photographs, etc.

[0030] Therefore, the object inventory manager module 120 can generate recommendations based on user profiles, such as generating digital marketing content about products or services based on the size, build, and style described in the user profile. In this way, the object recognition module can improve accuracy and thus improve the computational efficiency of generating recommendations based on the camera platform 118. In the following description regarding... Figures 7 to 10 Let me describe this example further.

[0031] Generally, the functions, features, and concepts described above and below can be used within the context of the example processes described in this section. Furthermore, the functions, features, and concepts described with respect to the different figures and examples in this document are interchangeable and are not limited to implementations within the context of a particular figure or process. Additionally, the boxes associated with the different representative processes and corresponding figures herein can be applied together and / or combined in different ways. Therefore, the various functions, features, and concepts described with respect to the different example environments, devices, components, figures, and processes herein can be used in any suitable combination and are not limited to the specific combinations represented by the examples listed in this specification.

[0032] Camera platform and object inventory control

[0033] Figure 2 The example implementation of system 200 is depicted, and system 200 is shown in more detail. Figure 1 Operation of the camera platform manager module 116. Figure 3 An example of user interaction with the camera platform manager module 120 for collecting metadata related to objects identified in digital images is depicted, as shown using the first, second, and third phases 302 to 306. Figure 4 This section describes an example of the identified objects and the corresponding metadata and storage results of user-defined object collection. Figure 5 An example implementation of a user interface configured to support the purchase and sale of identified objects using the Camera Platform Manager module 116 is described. Figure 6 It also describes how the output of the camera platform manager module is used to provide options for buying or selling objects identified from digital images.

[0034] The following discussion describes the techniques that can be implemented using the systems and devices described above. Figure 2 The various aspects of the process illustrated by the module can be implemented in hardware, firmware, software, or a combination thereof. The process is shown as a set of boxes, which specify operations performed by one or more devices and are not necessarily limited to the order in which the operations are performed for each box. In the sections discussed below, reference will be made to… Figures 2 to 6 .

[0035] Initially, a digital image 114 is acquired via the camera platform manager module 116. For example, a digital camera can be used to capture the digital image 114, such as a screenshot captured from the frame buffer of the computing device 102.

[0036] Then, the object recognition module 202 processes the digital image 114 to identify objects within the digital image 114. For example, the object recognition module 202 may employ a machine learning module 204 configured to use model 206, which can be used to identify objects using machine learning techniques such as neural networks, convolutional neural networks, deep learning networks, structured vector machines, decision trees, etc. For example, model 206 can be trained using training digital images labeled with corresponding identifiers. In this implementation, these training digital images and labels are obtained from a commercial service provider system, which is used by the vendor to add labels. Therefore, compared to conventional manual labeling techniques, a large number of accurately labeled training digital images can be obtained with minimal computational and user costs. Although shown as being implemented locally by computing device 102, this functionality can also be implemented, wholly or partially, by service provider system 104 via network 106.

[0037] Therefore, object identification data 208 describes the objects included in the digital image 114. Then, object data collection module 210 is used to collect object metadata 212 related to the identified objects. This can be performed locally via a search of local storage devices and / or remotely via network 106 through interaction with the service manager module 122 of the service provider system 104.

[0038] Various types of object metadata 212 can be obtained from various types of service provider systems 104. In one example, service provider system 104 provides information related to the purchase or sale of an object, such as product name, product description, and the price at which it was purchased or sold (e.g., based on an online auction). In another example, service provider system 104 provides information related to customer reviews of a product, such as star ratings or other ratings, text reviews, etc. In yet another example, the metadata describes replacement parts of an object, such as a filter, battery, light bulb, etc. Then, in this case, the object metadata 212 can be used, for example, to order these replacement parts in an efficient and intuitive way by selecting AR digital content formed from the metadata.

[0039] Then, in this example, object metadata 212 is provided to augmented reality (AR) configuration module 214. For example, AR configuration module 214 can be configured to generate AR digital content 126 based on object metadata 212 for display near objects via AR rendering module 216 for output devices 218, such as display devices, audio output devices, haptic (i.e., touch) output devices, etc.

[0040] To generate tactile effects, many output devices utilize some type of actuator or tactile output device. Examples of known tactile output devices for this purpose include electromagnetic actuators, such as an eccentric rotor motor (“ERM”) in which an eccentric mass moves via a motor, a linear resonant motor (“LRA”) in which a mass attached to a spring is driven back and forth, or “smart materials” such as piezoelectric, electroactive polymers, or shape memory alloys. Tactile output devices also broadly include non-mechanical or non-vibrational devices, such as those using electrostatic friction (ESF), ultrasonic surface friction (USF), or devices that utilize ultrasonic tactile transducers to induce acoustic radiation pressure, or devices that use a tactile substrate and a flexible or deformable surface, or devices that provide projected tactile output, such as a burst of air using an air jet, etc.

[0041] In this example, augmented reality content is both a direct view of the physical environment and... Figure 1 The content it supports is also a visual representation of the recreated physical environment. Figure 1 It supports content. In this way, through the use of camera platform 118, such as that implemented by camera platform manager module 116, users can easily “look around” using live feeds of digital image 114, select objects in digital image 114, and obtain metadata related to the objects.

[0042] In the replacement example, object recognition can be used to first identify the object. This identification can then be used as a "lookup" to locate the replacement part associated with the object, such as a filter, light bulb, battery, etc. AR digital content can then be output, selectable to purchase these items in a direct view of the user interface. In this example, this information is linked to past purchase history, allowing the AR digital content to indicate "when" to replace the replacement part, when the replacement part was most recently purchased, when it should be replaced, etc.

[0043] Figure 3 An example implementation 300 depicts a user interacting with a camera platform 118, such as that implemented by the camera platform manager module 116. This implementation 300 is shown using a first stage 302, a second stage 304, and a third stage 306.

[0044] At the first stage 302, the user interface 308 is output by the output device 218, such as a touch screen display device. In this example, the user interface 308 is configured as a "live feed" of digital images 114 acquired in real time from the digital camera 112.

[0045] At the second stage 304, user input is received, which selects an object displayed in the user interface 308. In the example shown, the user input is detected as a tap of the user's hand 318's finger detected using the touchscreen functionality of the output device 218. Other examples are also envisioned, such as a click and drag operation specifying a rectangular area, etc. In this way, the user can distinguish between multiple objects displayed simultaneously in the user interface 308. Other examples are also envisioned, such as spoken words or other gestures.

[0046] In response to the user selection in the second phase 304, the digital image 114 displayed in the user interface 308 is captured (e.g., obtained from the frame buffer) along with an indication of the location of a specific object, such as an X / Y coordinate guided by a “tap”. The digital image 114 is then processed by the object recognition module 202 as described above to identify the object (e.g., the can in the example shown) as object recognition data 208.

[0047] Then, in this example, object identification data 208 is transmitted to service provider system 104, which is configured to support the purchase and sale of goods. Therefore, in this example, service manager module 122 searches storage device 124 to obtain object metadata 212 related to the identified object. The object metadata 212 is then configured via AR configuration module 214 to generate AR digital content 126 for live / real-time feed output of digital images.

[0048] As shown at stage 306, an example of AR digital content 126 includes the name and price 310 of an object (e.g., average price, selling price, purchase price, etc.), which is displayed near the object, such as a can. AR rendering module 216 can then configure this content to remain “as is” relative to the view of the object, for example, based on data received from orientation sensors such as accelerometers, inertial devices, gyroscopes, image processing from digital images, and feature recognition.

[0049] AR digital content 126 also includes information about identical or similar goods 312, 314, 316, which can be purchased from service provider system 104, for example, as part of an online auction to set a price. In this way, camera platform manager module 116 implements camera platform 118 as non-modal within user interface 308, allowing the user to remain within the context of a real-time “live” feed of digital images and still have access to metadata describing the objects included in those digital images. For example, user 108 can “look around” at different objects in living room 110 and easily determine how to buy or sell these objects based on real-time information obtained from service provider system 104.

[0050] Figure 4 Another example implementation 400 of user interface 306 is depicted, which is configured to store and manage an inventory of objects identified using the techniques described above. User interface 308 includes thumbnails of digital images 402, 404, 406, which have been previously processed using the techniques described above and are “stored” in response to user interaction with user interface 306. For example, the thumbnails of digital images 402, 404, 406 indicate future interest in the various objects included in digital images 402 to 406.

[0051] In this example, the object inventory manager module 120 is configured to update the metadata associated with these objects through communication with the service provider system 104. This communication can follow a "push" model in response to price changes, implemented at regular intervals, or a "pull" model in response to accessing the screen of user interface 308.

[0052] In this example, the metadata includes the current individual names and prices 408, 410, 412, the user-specified set of objects and associated metadata 414 (e.g., the current value of the set), and the overall current value 416 of each of the saved objects. Therefore, users can form sets of saved objects as needed and interact with these sets as required, such as generating automatic lists together to sell the sets individually or as a whole.

[0053] Figure 5An example implementation 500 of user interface 308 is depicted, configured to buy and sell objects using saved objects identified and processed by a camera platform. In this example, the user selects to sell items corresponding to the jars and vases in digital images 402 and 404, and to buy another lamp corresponding to digital image 406. In response, object inventory manager module 120 collects metadata describing the corresponding prices of the objects sold 408 and 410, and purchased 502. Metadata describing the total value 504 of the items to be purchased and the total value 506 of the items to be sold is also generated and output to user interface 308. The overall change in values ​​508 is also output to user interface 308.

[0054] For example, a user might want to buy another lamp 502 and therefore sell some objects to cover the cost. In this example, by using objects saved by the platform, the user can quickly and efficiently determine how to achieve this, which would be impossible in potentially cumbersome conventional technologies.

[0055] Figure 6 Another example implementation 600 depicts using a camera platform to assist in the purchase or sale of objects identified in digital images collected from a digital camera. In this example, the user interface 308 also displays a live feed of digital images, including objects in the physical environment, such as the objects themselves, physical photographs of the objects, etc. In response, the object inventory manager module 120 includes options for the user to select to buy 602 or sell 604 products such as laundry detergent.

[0056] The object inventory manager module 120 also identifies objects that have been previously purchased through object recognition. This can be done based on local data from the computing device 102 and / or through communication with the service provider system 104. In response, AR digital content 126 is rendered in the user interface 308 as an option for a "repurchase" 606 to purchase a subscription at a discounted price. Thus, in this example, the user 108 can easily navigate the physical environment and purchase goods as needed in an efficient and intuitive way.

[0057] Other examples are also envisioned. In a first example, a user could capture a digital image of an object for which an upgraded model is currently unavailable but will become available in the near future. Therefore, metadata could describe the availability of the new model as well as the option to sell the user's current model. In a second example, metadata would be information such as when to replace the battery in a smoke detector, or cost savings that could be achieved by switching to a newer version of the product (e.g., an LED light that saves 85% of energy).

[0058] Camera platform and user profile

[0059] Figure 7 System 700 in the example implementation is depicted, showing a more detailed adoption of the user profile. Figure 1 Operation of the camera platform manager module 116. Figure 8 An example implementation 800 is described, which is at least partially based on machine learning for generating user profiles. Figure 9 Depicting based on Figure 9 The generated user profiles provide a recommended example system.

[0060] The following discussion describes the techniques that can be implemented using the systems and devices described above. (As described by...) Figure 8 and Figure 9 The various aspects of the process illustrated by the module can be implemented in hardware, firmware, software, or a combination thereof. The process is shown as a set of boxes, which specify operations performed by one or more devices and are not necessarily limited to the order in which the operations are performed for each box. In the sections discussed below, reference will be made to… Figures 7 to 9 .

[0061] In this example, the object inventory manager module 120 utilizes the camera platform 118 to make recommendations for user 108. For example, object recognition implemented using machine learning can also be used by the object inventory manager module 120 to process digital image 114. In this example, digital image 114 is used to generate a profile (e.g., user profile 702) based on features learned from digital image 114. This profile 702 is then used to form recommendations (e.g., through information such as...). Figure 8 The foundation for further description of machine learning, such as configuring digital marketing content with product recommendations based on these features.

[0062] For example, the profile may include digital images taken from user 704. Thus, the object inventory manager module 120 can determine the possible dimensions (e.g., size), body type (e.g., how the user wears clothing, such as tight, loose, or how the clothing is "suspended" on the user), style (e.g., professional, hippie, grunge, medieval), etc., of user 704. Digital images 114 may also be collected from sources other than the user but which the user desires, such as other people in person, physical photographs, etc.

[0063] Therefore, the object inventory manager module 120 can generate recommendations based on user profiles, such as generating digital marketing content about products or services based on the size, build, and style described in the user profile. In this way, the object recognition module can improve accuracy and thus improve the computational efficiency of generating recommendations based on the camera platform 118.

[0064] Figure 8 A more detailed description Figure 1An example of the generation of user profile 702. Beginning, as previously described, digital image 114 is obtained by camera platform manager module 116. For example, digital image 114 can be captured using a digital camera, as a screenshot captured from the frame buffer of computing device 102, etc.

[0065] Then, the object recognition module 202 processes the digital image 114 to identify objects within the digital image 114. For example, the object recognition module 202 may employ a machine learning module 204 configured to use model 206, which can be used to identify objects using machine learning, such as a neural network. For example, model 206 can be trained using training digital images labeled with corresponding identifiers. In this implementation, these training digital images and labels are obtained from a business service provider system and are labeled by the vendor using that system. Therefore, compared to conventional manual labeling techniques, a large number of training digital images can be obtained with minimal computational and user costs. Although shown as being implemented locally by computing device 102, this functionality can also be implemented, wholly or partially, by service provider system 104 via network 106.

[0066] Therefore, object recognition data 208 describes the objects included in the digital image 114. Machine learning module 204 and the corresponding model are also trained to identify possible sizes, body types, and styles presented in the digital image 114, and thus output size data 802, body type data 804, and style data 806 as part of the object recognition data 208. Size data 802 may include the possible overall size of user 704, the possible clothing size worn by the user, etc. Body type data 804 describes how user 704 wears clothing, for example, oversized, tight, sporty, etc. For example, body type data 804 can be determined by comparing the skeletal tracking of user 704 (e.g., using a structured light camera or time-of-flight camera) with the overall size of the clothing—e.g., "how the clothing hangs on the user," such as loose as illustrated. Style data 806 describes the style of user 704's clothing and / or user 704 itself.

[0067] Digital images 114 can also be captured from other users with preference characteristics, such as people on the street wearing desired jackets, digital images taken from physical photographs of pop singers in physical magazines, etc. Therefore, this object recognition data 208 is collected by the profile data collection module 808 to generate user profile data 810 describing the characteristics of users and / or other users identified from digital image 114.

[0068] like Figure 9As shown, user profile 702 can then be transmitted to service provider system 104, which generates recommendation 902 based on the user profile. For example, recommendation 902 can be generated using machine learning based on user segments identified through nonnegative matrix factorization. In this way, the camera platform 118 of object inventory manager module 120 can address potential user needs based on digital images and object recognition supported by the platform. Although described as being implemented by computing device 102, this functionality can also be implemented, in whole or in part, by service provider system 104, for example, in response to images transmitted as part of a social networking service.

[0069] Example systems and devices

[0070] Figure 10 An example system is illustrated generally as 1000, which includes an example computing device 1002 representing one or more computing systems and / or devices that can implement the various technologies described herein. This is illustrated by including a camera platform manager module 116. For example, computing device 1002 may be a server of a service provider, a device associated with a client (e.g., a client device), a system-on-a-chip, and / or any other suitable computing device or computing system.

[0071] The example computing device 1002 shown includes a processing system 1004 communicatively coupled to each other, one or more computer-readable media 1006, and one or more I / O interfaces 1008. Although not shown, the computing device 1002 may also include a system bus or other data and command transfer system that couples the various components to each other. The system bus may include any one or a combination of different bus architectures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus utilizing any of the various bus architectures. Various other examples are also contemplated, such as control lines and data lines.

[0072] Processing system 1004 represents the functionality of using hardware to perform one or more operations. Therefore, processing system 1004 is shown as including hardware elements 1010 that can be configured as processors, function blocks, etc. This can include implementations as hardware methods of application-specific integrated circuits or other logic devices formed using one or more types of semiconductors. Hardware element 1010 is not limited by the materials formed therein or the processing mechanism employed therein. For example, a processor may include one or more semiconductors and / or transistors (e.g., integrated circuits (ICs)). In such a case, processor-executable instructions may be electronically executable instructions.

[0073] Computer-readable storage medium 1006 is shown as including memory / storage device 1012. Memory / storage device 1012 represents the capacity of memory / storage device associated with one or more computer-readable media. Memory / storage device component 1012 may include volatile media (e.g., random access memory (RAM)) and / or non-volatile media (e.g., read-only memory (ROM), flash memory, optical disk, magnetic disk, etc.). Memory / storage device component 1012 may include fixed media (e.g., RAM, ROM, fixed hard disk drive, etc.) and removable media (e.g., flash memory, removable hard disk drive, optical disk, etc.). Computer-readable medium 1006 may be configured in various other ways as further described below.

[0074] One or more input / output interfaces 1008 represent functions that allow a user to input commands and information into the computing device 1002 and also allow information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include keyboards, cursor control devices (e.g., mice), microphones, scanners, touch functionality (e.g., capacitive sensors or other sensors configured to detect physical touch), cameras (e.g., which may employ visible or invisible wavelengths, such as infrared frequencies, to identify movement as a gesture not involving touch), etc. Examples of output devices include display devices (e.g., monitors or projectors), speakers, printers, network interface cards, haptic-responsive devices, etc. Therefore, the computing device 1002 can be configured to support user interaction in various ways as further described below.

[0075] This document describes various techniques in the general context of software, hardware components, or program modules. Typically, such modules include routines, programs, objects, elements, components, data structures, etc., that perform specific tasks or implement specific abstract data types. As used herein, the terms “module,” “function,” and “component” generally refer to software, firmware, hardware, or a combination thereof. The techniques described herein are platform-independent, meaning they can be implemented on a variety of commercial computing platforms with various processors.

[0076] Implementations of the described modules and technologies may be stored on or transmitted on some form of computer-readable medium. The computer-readable medium may include various media accessible to the computing device 1002. By way of example, and not limitation, the computer-readable medium may include "computer-readable storage media" and "computer-readable signal media".

[0077] Compared to mere signal transmission, carrier waves, or signals themselves, "computer-readable storage media" can refer to media and / or devices capable of enabling permanent and / or non-transitory storage of information. Therefore, computer-readable storage media refers to non-signal-bearing media. Computer-readable storage media include hardware such as volatile and non-volatile, removable and non-removable media and / or storage devices implemented with methods or techniques suitable for storing information such as computer-readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disk (DVD) or other optical storage devices, hard disks, magnetic tape cassettes, magnetic tapes, disk storage devices or other magnetic storage devices, or other storage devices, tangible media, or manufactured products suitable for storing desired information and accessible by a computer.

[0078] "Computer-readable signal medium" can refer to a signal-bearing medium configured to transmit instructions to hardware of computing device 1002, for example, via a network. Signal media typically implement computer-readable instructions, data structures, program modules, or other data in modulated data signals such as carrier waves, data signals, or other transmission mechanisms. Signal media also includes any information transmission medium. The term "modulated data signal" means a signal having one or more characteristics that are set or changed in a manner that encodes information as a signal. By way of example, and not limitation, communication media include wired media such as wired networks or direct wired connections, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0079] As previously described, hardware element 1010 and computer-readable medium 1006 represent modules, programmable device logic, and / or fixed device logic implemented in hardware form. This hardware form can be employed in some embodiments to implement at least some aspects of the techniques described herein, such as to execute one or more instructions. The hardware may include components of an integrated circuit or system-on-a-chip, application-specific integrated circuit (ASIC), field-programmable gate array (FPGA), complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this context, the hardware may operate as a processing device to perform program tasks defined by instructions and / or logic implemented by the hardware, as well as hardware for storing instructions for execution, such as the previously described computer-readable storage medium.

[0080] The foregoing combinations can also be used to implement the various techniques described herein. Therefore, software, hardware, or executable modules can be implemented as one or more instructions and / or logic implemented on some form of computer-readable storage medium and / or by one or more hardware elements 1010. Computing device 1002 can be configured to implement specific instructions and / or functions corresponding to the software and / or hardware modules. Thus, the implementation of modules executable as software by computing device 1002 can be at least partially implemented in hardware, for example, by using the computer-readable storage medium and / or hardware elements 1010 of processing system 1004. Instructions and / or functions can be executable / operable by one or more manufactured products (e.g., one or more computing devices 1002 and / or processing systems 1004) to implement the techniques, modules, and examples described herein.

[0081] The techniques described herein can be supported by various configurations of computing device 1002, and are not limited to specific examples of the techniques described herein. This functionality can also be implemented, in whole or in part, via a distributed system such as, through platform 1016 via “cloud” 1014 as described below.

[0082] Cloud 1014 includes and / or represents platform 1016 for resource 1018. Platform 1016 abstracts the underlying functionality of the hardware (e.g., server) and software resources of cloud 1014. Resource 1018 may include applications and / or data that can be utilized when computer processing is performed on a server remote from computing device 1002. Resource 1018 may also include services provided via the Internet and / or via user networks such as cellular or Wi-Fi networks.

[0083] Platform 1016 can abstract resources and functions to connect computing device 1002 to other computing devices. Platform 1016 can also be used to abstract resource scaling to provide appropriate scaling levels to meet the demands encountered by resource 1018 implemented via platform 1016. Therefore, in the interconnect device implementation, the implementation of the functions described herein can be distributed throughout system 1000. For example, the function can be implemented partly on computing device 1002 and partly via platform 1016, which abstracts the functions of cloud 1014.

[0084] in conclusion

[0085] Although the invention has been described in language specific to structural features and / or methodological actions, it should be understood that the invention as defined in the appended claims is not necessarily limited to the specific features or actions described. Rather, the specific features and actions are disclosed as exemplary forms for implementing the claimed invention.

Claims

1. A system for augmented reality, comprising: A camera platform manager module, which is at least partially implemented in the hardware of at least one computing device, is configured to perform the following operations: Output a live feed of digital live images taken from the physical environment to the user interface; as well as The user interface receives user selections of objects included in the digital live image. An object recognition module that uses machine learning to identify selected objects is implemented at least partially in the hardware of the at least one computing device; An object data collection module is implemented, at least in part, in the hardware of the at least one computing device, to obtain metadata relating to the identified object, the metadata including the price at which the identified object is sold; An augmented reality configuration module, which is implemented at least in part in the hardware of the at least one computing device, is configured to generate augmented reality digital content at least in part based on the obtained metadata; as well as An augmented reality rendering module, which is implemented at least in part in the hardware of the at least one computing device, is configured to display in the user interface: a thumbnail of the identified object generated from the digital live image, a first representation of the price at which the identified object is sold, indicated by the augmented reality digital content, a thumbnail of additional identified objects in the inventory of the identified object, a second representation of the price at which the additional identified objects are sold, and a third representation of the price at which all identified objects are sold.

2. The system according to claim 1, wherein, The metadata also describes the purchase price of the identified object from the service provider system that provided the metadata.

3. The system according to claim 2, wherein, The price is based on multiple online auctions involving the identified object.

4. The system according to claim 1, wherein, The metadata includes information describing the characteristics of the identified object.

5. The system according to claim 1, wherein, The metadata includes information describing the replacement of parts of the identified object.

6. The system according to claim 5, wherein, The part in question is either a battery or a filter.

7. The system according to claim 1, wherein, The metadata includes the option to purchase another instance of the identified object.

8. The system according to claim 1, wherein, The augmented reality rendering module is further configured such that the augmented reality digital content is displayed within the live feed of the digital live image, and that the augmented reality digital content is displayed near the identified object.

9. The system according to claim 1, wherein, The augmented reality rendering module is also configured such that the augmented reality digital content is output together with a digital live image of the physical environment.

10. A method for augmented reality implemented by a computing device, the method comprising: The computing device outputs live feeds of digital live images captured from the physical environment to the user interface; The computing device receives user selections of objects included in the digital live image; The computing device uses machine learning to identify the selected objects; The computing device obtains metadata, which includes the option to purchase another instance of the identified object, and also includes the price at which the identified object is sold. Augmented reality digital content is generated by the computing device based at least in part on the obtained metadata; as well as The computing device displays in the user interface: a thumbnail of the identified object generated from the digital live image, a first representation of the price at which the identified object is sold, indicated by the augmented reality digital content, a thumbnail of additional identified objects in the inventory of the identified object, a second representation of the price at which the additional identified objects are sold, and a third representation of the price at which all identified objects are sold.

11. The method according to claim 10, wherein, The metadata also includes the purchase price of the identified object from the service provider system that provided the metadata.

12. The method according to claim 10, wherein, The metadata includes information describing the characteristics of the identified object.

13. The method according to claim 10, wherein, The metadata includes information describing the replacement of parts of the identified object.

14. The method according to claim 13, wherein, The part in question is either a battery or a filter.

15. The method of claim 10, further comprising: The augmented reality digital content is displayed within the live feed of the digital live image, wherein the augmented reality digital content is displayed near the identified object.

16. The method of claim 10, further comprising: The augmented reality digital content is output together with the digital live image.

17. A system for augmented reality, comprising: A camera platform manager module, which is at least partially implemented in the hardware of at least one computing device, is configured to perform the following operations: Output a live feed of digital live images taken from the physical environment to the user interface; as well as The user interface receives user selections of objects included in the digital live image. An object recognition module that uses machine learning to identify selected objects is implemented at least partially in the hardware of the at least one computing device; An object data collection module is implemented, at least in part, in the hardware of the at least one computing device to obtain metadata related to the identified object, the metadata related to the identified object including the price at which the identified object is sold and metadata of the replacement portion of the identified object; as well as An augmented reality configuration module, which is implemented at least in part in the hardware of the at least one computing device, is configured to generate augmented reality digital content at least in part based on the obtained metadata; as well as An augmented reality rendering module, which is implemented at least in part in the hardware of the at least one computing device, is configured to display in the user interface: a thumbnail of the identified object generated from the digital live image, a first representation of the price at which the identified object is sold, indicated by the augmented reality digital content, a thumbnail of additional identified objects in the inventory of the identified object, a second representation of the price at which the additional identified objects are sold, and a third representation of the price at which all identified objects are sold.

18. The system according to claim 17, wherein, The replacement part is a battery or a filter.

19. The system according to claim 17, wherein, The augmented reality rendering module is further configured such that the augmented reality digital content is displayed within the live feed of the digital live image, and that the augmented reality digital content is displayed near the identified object.

20. The system according to claim 17, wherein, The price of the identified object is the selling price of the identified object from the service provider system that provides metadata related to the identified object.

Images