Augmenting reality by capturing signals using radios

Abstract

In one example, a method may include receiving a wireless signal from a device. The device may be in an environment. The method may also include identifying the device. The device may be identified based on at least one of data and metadata included in the wireless signal. Further, the method may include determining a location of the device within the environment. The location of the device within the environment may be determined based on the data included in the wireless signal. Further, the method may include displaying an augmented environment representing the environment and a symbol representing the device within the augmented environment. The symbol may be displayed in a location within the augmented environment corresponding to the location of the device within the environment.

Description

[Background technology]

[0001] The present disclosure relates generally to augmenting reality. In particular, the present disclosure describes systems and methods for augmenting reality by capturing signals using radios.

[0002] Augmented reality (AR) and virtual reality (VR) include practices that augment the real world or replace it with simulated elements or an entire simulated world. Additionally, AR and VR may be used to enhance a user's perceived reality with additional graphics, text, sound, video, etc. For example, AR and / or VR may add game characters, avatars of other users, etc. to a simulated world. Thus, the present disclosure relates to augmenting reality by identifying devices by capturing signals using radios.

[0003] The claimed subject matter is not limited to embodiments that solve the disadvantages or operate only in environments as described above. This background is only provided to illustrate examples where the present disclosure may be utilized. Summary of the Invention

[0004] The present disclosure relates generally to augmenting reality. In particular, the present disclosure describes systems and methods for augmenting reality by identifying devices by capturing signals using radios.

[0005] In one example, a method may include receiving a wireless signal from a device. The device may be within an environment. The method may also include identifying the device. The device may be identified based on at least one of data and metadata included in the wireless signal. Further, the method may include determining a location of the device within the environment. The location of the device within the environment may be determined based on the data included in the wireless signal. Further, the method may include displaying an augmented environment representing the environment and a symbol representing the device within the augmented environment. The symbol may be displayed in a location within the augmented environment corresponding to the location of the device within the environment.

[0006] In another example, a method may include receiving a first wireless signal from a first device. The first device may be within an environment. The method may also include receiving a second wireless signal from a second device. The second device may also be within the environment. Further, the method may include identifying the first device. The first device may be identified based on at least one of data and metadata included in the first wireless signal. Further, the method may include identifying the second device. The second device may be identified based on at least one of data and metadata included in the second wireless signal. The method may include determining locations of the first device and the second device within the environment. The locations of the first device and the second device may be determined based on the first wireless signal and the second wireless signal. The method may also include displaying an augmented environment. The augmented environment may represent the environment and may include symbols representing the first device and the second device. The symbols may be displayed in locations within the augmented environment corresponding to the locations of the first device and the second device within the environment.

[0007] This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential characteristics of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 illustrates an exemplary network architecture. [Figure 2] FIG. 1 illustrates an exemplary environment for augmenting reality by capturing signals using radios. [Figure 3] 1 is a flow diagram of an exemplary method for augmenting reality by capturing signals using a radio. [Figure 4] FIG. 1 shows a diagrammatic representation of a machine in the example form of a computing device within which a set of instructions for causing the machine to perform any one or more of the methods described herein may be executed. DETAILED DESCRIPTION OF THE INVENTION

[0009] Reference will be made to the drawings and specific language will be used to describe various aspects of the present disclosure. Use of the drawings and description in this manner should not be construed as limiting the scope of the present disclosure. Additional aspects may become apparent in light of this disclosure, including the claims, or may be learned by practice.

[0010] As the number of wireless devices using standardized communication protocols (e.g., Bluetooth®) increases and the cost of manufacturing wireless devices decreases, the potential for augmenting reality using wireless signals transmitted and received by these devices also increases. Furthermore, as the number of wireless devices increases, so do the security and privacy issues associated with these devices, particularly as the environment becomes increasingly populated with wireless devices (e.g., IoT devices), for example, in commercial and industrial applications.

[0011] The descriptions contained within this disclosure are generally provided using examples of augmented environments. It should be understood that an augmented environment can apply to augmented reality (AR), which involves enhancing a real environment (generally referred to as an "environment" in this disclosure) with additional information and / or visuals, and to virtual reality (VR), which involves replacing a view of a real environment with a simulated environment.

[0012] In some embodiments, to generate an augmented environment, devices in the environment (e.g., computing devices and / or sensors) may transmit wireless signals including identifying information. For example, the data included in the wireless signals may include identification information, location information, device / object characteristic descriptors, or any other suitable type of data. Smart devices may receive wireless signals from various devices. Smart devices may include smartphones, computing devices, televisions, digital video disc (DVD) players, VR headsets, cell phones, tablets, personal computers, or any other suitable devices.

[0013] The smart device may identify one or more of the devices in the environment using wireless signals. In some embodiments, the identification process may include determining the type of each of the devices transmitting the wireless signals. Further, in some embodiments, one or more of the devices may include a sensor physically attached to an object that is not configured to transmit wireless signals. For example, a sensor may be physically attached to a sofa located in a room of a residential building. The sensor may transmit a wireless signal that includes data related to the corresponding object. For example, the data may indicate that the object is a sofa and the sofa's corresponding dimensions. In these and other embodiments, the identification process may include determining the type of each of the objects associated with the sensor.

[0014] In at least one embodiment, the data may indicate purchasing information about the object. For example, in the case of a sofa, the data may indicate price, availability, delivery time, services, etc. related to the sofa. In a further example, a hotel room may include several objects, each transmitting a wireless signal including data related to the corresponding object. For example, a hotel room may include a television, a sofa, a mattress, a hair dryer, a chair, a desk, etc. Each of these items may transmit a wireless signal including data of any type of information related to the corresponding product. Furthermore, the data may include purchasing information about any or all of the objects. In one example, a user may use a user device to receive and view data about objects in a hotel room. The user may purchase an item, such as, for example, a new mattress that is the same model as the mattress in the hotel room. The user may select delivery of a mattress at the user's home, and the mattress may be delivered and ready for the user when the user returns home from the hotel. In yet a further example, the user may purchase the object itself or a service for the object, which may include any service. For example, the user may purchase installation service or a warranty plan for the object.

[0015] The smart device may also determine the location of one or more of the devices and / or objects within the environment. In some embodiments, at least some of the wireless signals may include data indicating the physical location of the corresponding device or object within the environment. In these and other embodiments, the smart device may be configured to determine the location of one or more of the devices or objects based on signal characteristics of the corresponding wireless signals. In some embodiments, the smart device may determine the physical location of one or more of the devices relative to each other. In these and other embodiments, the smart device may determine the physical location of one or more of the devices relative to the physical location of the smart device. Additionally or alternatively, the smart device may determine the physical location of one or more of the devices relative to an external boundary of the environment (e.g., relative to a physical boundary such as a wall, a building, a body of water, a painted line, or any other object that can create a boundary).

[0016] The smart device may include a display. The smart device may display the augmented environment via the display. The augmented environment may represent the environment. Additionally, the smart device may display symbols representing one or more devices and / or objects in the augmented environment. For example, the smart device may display symbols of a sofa, personal computer, television, smartphone, DVD player, storefront, building, vehicle, people, tools, medical device, park bench, landmark, camera, or any other suitable device or object. In some embodiments, the smart device may display symbols in locations in the augmented environment corresponding to the device's location in the environment. Additionally, the smart device may display additional information about the device and / or object. The additional information visible in the augmented environment may be more extensive than the information visible in the environment. For example, the additional information may include characteristics of the device and / or object, such as weight, height, age, medical history, attendance history, current sales, sales descriptions, goods and / or services offered, product descriptions, service descriptions, year of manufacture, make, model, or any other suitable information corresponding to the device and / or object. Including additional information in an augmented environment can improve the experience of a user of a smart device.

[0017] Accordingly, this disclosure describes systems and methods for augmenting reality. In particular, this disclosure describes systems and methods for augmenting reality by capturing signals using radios.

[0018] 1 illustrates an example network architecture 100 in which embodiments of the present disclosure may be implemented. Network architecture 100 may include one or more endpoint devices 105, one or more intermediate devices 115, one or more relay servers 125, and one or more endpoint manager servers 135. In some embodiments, network architecture 100 may be capable of moving data between one or more endpoint devices 105 and various endpoint manager servers 135 via crowdsourced intermediate devices 115, which may function as network clients, and one or more relay servers 125.

[0019] The endpoint device 105 may include one or more wireless devices (e.g., IoT devices). The endpoint device 105 may include a power source, a data collection device (e.g., a sensor), and a network device. The power source may include a battery or a connection to a power grid. Additionally or alternatively, the power source may include an energy harvesting device, such as a solar panel, a solar cell, photovoltaic power, electromagnetic, etc. In at least some embodiments, the endpoint device 105 may not include a power source and instead may use ambient backscattering techniques. The endpoint device 105 may also include one or more sensors. The one or more sensors may be configured to detect any type of condition and generate electronic data based on the detected condition. For example, the endpoint device 105 may include a smartwatch with a heart rate monitor configured to generate heart rate data using heart rate conditions collected by the heart rate monitor. In some embodiments, the endpoint device 105 does not have the capability to communicate over the Internet and includes only hardware and / or software (e.g., includes only sensors) capable of communicating with nearby devices, such as nearby intermediate device 115. In other embodiments, endpoint device 105 may include hardware and / or software for communicating over the Internet. In at least one embodiment, endpoint device 105 may include a hardware secure element (SE) capable of securely hosting applications and storing sensitive and cryptographic data.

[0020] The network devices of the endpoint devices 105 may include any hardware, software, or combination thereof capable of communicating with another device over a wireless network. In at least one embodiment, the network devices may include any network controller configured to communicate over a short-range network such as Bluetooth® or any other short-range network. In at least one embodiment, the network devices may include any network controller configured to communicate over a low-power network. Exemplary endpoint devices 105 include, but are not limited to, industrial devices, residential appliances, commercial appliances, inventory trackers, smart watches, wearables, heart rate monitors, logistics trackers, environmental sensors, cash registers, credit card readers, point-of-sale (POS), bicycles, electric scooters, electric skateboards, vehicles, cars, electric vehicles, satellites, smart devices, or any device (mobile and non-mobile including a wireless interface). The network architecture 100 may include any number of endpoint devices 105, and the endpoint devices 105 in the network architecture 100 may be any type of endpoint device 105, including any type of network-enabled device. In some embodiments, the endpoint devices 105 may be fixed or relatively stationary in the network architecture 100, such as a point of sale, a pollution sensor, a sensor attached to a stationary object, or a personal computer. Additionally or alternatively, the endpoint devices 105 may be mobile, such as a smart watch, a smartphone, a laptop, a tablet, or any car or vehicle.

[0021] One or more endpoint devices 105 may be configured to communicate with other devices over at least one wireless network 110. For example, a first endpoint device 105a may be in electronic communication with a first intermediate device 115a over the wireless network 110a. The one or more intermediate devices 115 may include any type of device capable of communicating with the endpoint devices 105 over the wireless network 110 and with the relay server 125 over the second network 120. In at least one embodiment, the intermediate device 115 may include two network controllers: a first network controller for communicating over the wireless network 110 and a second network controller for communicating over the second network 120. Exemplary intermediate devices 115 include a mobile device, a personal computer (PC), a laptop, a smartphone, a netbook, an e-reader, a personal digital assistant (PDA), a cellular phone, a mobile phone, a tablet, a vehicle, a drone, a car, a truck, a wearable device, glasses, a router, a television, a set-top box, or the like.

[0022] As shown, a first endpoint device 105a may be in electronic communication with a first intermediate device 115a over a wireless network 110a (e.g., a short-range network). Additionally, a second endpoint device 105b may be in electronic communication with a second intermediate device 115b over another wireless network 110b (e.g., a low-power network). A third endpoint device 105c may be in electronic communication with a third intermediate device 115c over another wireless network 110c. A fourth endpoint device 105d may be in electronic communication with a fourth intermediate device 115d over another wireless network 110d.

[0023] In some embodiments, the wireless network 110 may be any network that uses a relatively low amount of power. Exemplary wireless networks 110 may include any Bluetooth® network type (e.g., Bluetooth Low Energy (BLE), Bluetooth 4.0, Bluetooth 5.0, Bluetooth Long Range), NB-IoT, LTE Direct, LTE-M, LTE M2M, 5G, Wi-Fi, Wi-Fi Aware, mmWave, or any other type of network. One or more endpoint devices 105 may connect to various intermediate devices 115 using different types of wireless networks 110. For example, a first endpoint device 105a may be in electronic communication with a first intermediate device 115a via a first short-range wireless network 110a, and a second endpoint device 105b may be in electronic communication with a second intermediate device 115b via a second short-range wireless network 110b.

[0024] The endpoint device 105, the intermediate device 115, or both may be fixed, relatively stationary, or mobile. When the endpoint device 105 and the intermediate device 115 come within wireless range of each other, the endpoint device 105 and the intermediate device 115 may perform a handshake and / or authentication to initiate data exchange between the endpoint device 105 and the intermediate device 115.

[0025] In some embodiments, endpoint device 105 may periodically send a beacon (e.g., a radio signal) containing data over wireless network 110. Endpoint device 105 may include various services that may run on endpoint device 105. For example, a smartwatch may include a clock service, a heart rate monitor service, a motion detection service, a music service, etc. A beacon may be generated for each of these services, or a single beacon may be generated to include data for some or all of the services.

[0026] Intermediate device 115 may listen for such beacons from endpoint devices. In response to receiving the beacons, intermediate device 115 may send the beacons to relay server 125 via second network 120. In at least one embodiment, wireless network 110 and second network 120 are different types of networks. For example, wireless network 110 may be a Bluetooth network, and second network 120 may be a cellular network, Wi-Fi, or the Internet. In response to receiving the beacons, intermediate device 115 may also generate augmented reality or virtual reality elements for display at intermediate device 115. For example, in response to receiving a beacon from connected skateboard device 105, glasses 115 may present information about the skateboard via a display. For example, a user wearing glasses 115 may be walking outside, and another user passes by on the connected skateboard 105. Glasses 115 may provide information about the skateboard 115, such as via one or more augmented reality elements. Prior to providing the information, the glasses 115 may present a notification that, when activated by the wearer of the glasses 115, causes the glasses 115 to provide the information. The information may include purchase information such that the wearer of the glasses 115 can purchase the skateboard 115 and have it shipped to a location of the wearer's choosing.

[0027] The second network 120 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.xx network or a Wi-Fi network), a cellular network (e.g., a Long Term Evolution (LTE) or LTE-Advanced network, 1G, 2G, 3G, 4G, 5G, 6G, etc.), a router, a hub, a switch, a server computer, and / or combinations thereof.

[0028] The relay server 125 may send the beacon, or information related to the beacon, to the endpoint manager server 135 via the third network 130. The third network 130 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.xx network or a Wi-Fi network), a cellular network (e.g., a Long Term Evolution (LTE) or LTE-Advanced network, 1G, 2G, 3G, 4G, 5G, 6G, etc.), a router, a hub, a switch, a server computer, and / or combinations thereof. In at least one embodiment, the second network 120 and the third network 130 are the same network or include at least some overlapping components.

[0029] The one or more relay servers 125 may include one or more computing devices, such as a rackmount server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a smartphone, a car, a drone, a robot, any mobility device having an operating system, etc.), a data store (e.g., a hard disk, memory, a database), a network, software components, and / or hardware components. The one or more relay servers 125 may be configured to receive beacons from the intermediate device 115. The one or more relay servers 125 may send the beacons or data related to or associated with the beacons to the endpoint manager server 135. The one or more relay servers 125 may receive messages from the endpoint manager server 135 and, in some embodiments, send messages from the endpoint manager server 135 to the intermediate device 115. In at least some embodiments, the intermediate device 115 may perform one or more actions in response to receiving a message from the endpoint manager server 135. These actions may include actions local to the intermediate device 115 and / or sending messages from the endpoint manager server 135 to the endpoint device 105 .

[0030] The endpoint manager server 135 may include one or more computing devices, such as a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, a smartphone, a car, a drone, a robot, any mobility device with an operating system, etc.), a data store (e.g., a hard disk, memory, a database), a network, software components, and / or hardware components. The endpoint manager server 135 may be associated with one or more endpoint devices 105. For example, a particular company, person, or manufacturer may sell the endpoint devices 105 and use the endpoint manager server 135 to communicate with, control, or store data related to the endpoint devices 105.

[0031] The endpoint manager server 135 may send messages related to a particular endpoint device 105 or a set of endpoint devices 105. For example, the endpoint manager server 135 may send information describing the characteristics or features of a particular endpoint device 105 or a set of endpoint devices 105. Additionally, in some embodiments, the endpoint manager server 135 may send other communications to the endpoint devices 105, such as responses to requests from beacons generated by a particular endpoint device 105.

[0032] Each relay server 125 may include a message manager 140. The message manager 140 may be implemented using hardware, including a processor, a microprocessor (e.g., for performing or controlling the performance of one or more operations), an FPGA, or an ASIC. In some other cases, the message manager 140 may be implemented using a combination of hardware and software. A software implementation may include rapid activation and deactivation of one or more transistors or transistor elements, such as may be included in the hardware of a computing system (e.g., relay server 125). Additionally, software-defined instructions may operate on information within the transistor elements. Implementation of the software instructions may at least temporarily reconfigure electronic pathways and transform computing hardware.

[0033] Each relay server 125 may include data storage 145. Data storage 145 may include any memory or data storage. In some embodiments, data storage 145 may include a computer-readable storage medium for carrying or having computer-executable instructions or data structures stored thereon. A computer-readable storage medium may include any available medium that can be accessed by a general-purpose or special-purpose computer, such as a processor. For example, data storage 145 may include computer-readable storage media, including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices), or any other storage medium that can be used to carry or store desired program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. Combinations of the above may be included in data storage 145. In the illustrated embodiment, data storage 145 is part of relay server 125. In some embodiments, data storage 145 may be separate from relay server 125, and data storage 145 may be accessed over a network. In at least one embodiment, data storage 145 may include multiple data storages.

[0034] Data storage 145 may include data related to endpoint devices 105, intermediate devices 115, and endpoint manager servers 135, and relationships between endpoint devices 105, intermediate devices 115, and endpoint manager servers 135. For example, data storage 145 may include a table or list of endpoint devices associated with a particular endpoint manager server 135. The data storage 145 may include data related to beacons received from endpoint devices, such as a timestamp of receipt of the beacon, a timestamp associated with the creation of the beacon, a geolocation associated with the beacon and / or associated with the endpoint device 105 that created or sent the beacon, sensor data associated with the endpoint device 105, routing information about how and / or where to send data between the endpoint manager server 135 and the endpoint device 105, the strength of the connection between the intermediate device 115 and the endpoint device 105, the proximity of the endpoint device 105 to the intermediate device 115, the type of wireless network 110 connecting the intermediate device 115 to the endpoint device 105, the cost of the connection between the intermediate device 115 and the endpoint device 105, the current battery level of the intermediate device, and the type of intermediate device.

[0035] The message manager 140 may handle communications between the endpoint device 105, the intermediate device 115, and the endpoint manager server(s) 135. In one example, the message manager 140 may receive a beacon from the first intermediate device 115a via the second network 120. The beacon may have been sent by the endpoint device 105a to the first intermediate device 115a via the wireless network 110a. The beacon may contain characteristics about the endpoint device 105a, including an identifier (e.g., MAC address, unique ID) of the endpoint device 105a, the geographic location of the endpoint device 105a, and an advertisement of the UUIDs of services supported by the endpoint device 105a. The message manager 140 may identify the characteristics of the beacon, such as by analyzing the beacon to identify information relevant to the beacon. The message manager 140 may access the data storage 145 to identify the endpoint manager server 135 associated with the beacon based on the characteristics of the beacon. For example, the identifier of the endpoint device may be associated with a particular manufacturer that operates a particular endpoint manager server 135. Message manager 140 may identify this particular endpoint manager server 135 in data storage 145 and an address and / or route to send a beacon to reach endpoint manager server 135. In at least some embodiments, message manager 140 may send a beacon or beacon message to endpoint manager server 135 over third network 130. The beacon message may include a beacon, may not include a beacon, or may include information related to the beacon.

[0036] In at least one embodiment, a beacon may include data from multiple services associated with the endpoint device 105. Additionally or alternatively, multiple beacons from a single endpoint device 105 may be generated and broadcast over the wireless network 110. Each of these multiple beacons may be associated with a different service associated with the endpoint device 105, for example. Alternatively, each of these multiple beacons may be associated with a different network type of the wireless network 110, for example. The message manager 140 may identify the service and / or network type and, based on the information about the service, identify the appropriate endpoint manager server 135 that should receive the beacon message.

[0037] The endpoint manager server 135 may receive messages from the relay server 125. The endpoint manager server 135 may store the messages, process the messages, generate reports based on the messages, generate notifications or responses based on the messages, or any other action based on the messages. For example, the endpoint manager server 135 may generate a response message related to the beacon message. The response message may include a message targeted to one or more of the relay server 125, the intermediate device 115, the endpoint device 105 that generated the beacon, or another endpoint device 105 that did not generate the beacon. The endpoint manager server 135 may send the response message to the same relay server 125 that sent the beacon message to the endpoint manager server 135 (e.g., relay server 125a) or to a different relay server 125 that did not send the beacon message to the endpoint manager server 135 (e.g., relay server 125b).

[0038] The relay server 125 may receive a response message related to the beacon message from the endpoint manager server 135. The relay server 125 may process the response message by performing an action at the relay server 125, sending data to another device (e.g., a user device), sending data to the endpoint device 105, etc.

[0039] In some embodiments, network architecture 100 can be used to exchange data between any devices capable of network-based communication over the Internet. In one example, network architecture 100 can leverage existing smartphone infrastructure to create delay-tolerant connectivity. Network architecture 100 can move data to the cloud in an early, delay-tolerant manner, which can be useful for many types of wireless communications (e.g., IoT communications), such as firmware updates, status updates, log file storage, and micropayments. Intermediate devices 115 can include software running on smartphones to periodically scan for other devices (e.g., endpoint devices 105), such as industrial devices, smart watches, wearables, logistics trackers, and environmental sensors. These endpoint devices 105 can connect with software clients running on smartphones to create a large-scale, wide-area network for moving data to and within the cloud.

[0040] Furthermore, it is estimated that 95% of the human population is covered by some cellular service. Network architecture 100 can be deployed anywhere in the world, allowing areas with lower connectivity to increase their connectivity. Moreover, network architecture 100 can provide coverage beyond the range of traditional cellular networks, for example, by using software running on a Bluetooth-enabled smartphone. A user may travel to an area with limited or no cellular connectivity but still receive beacons from endpoint device 105 via wireless network 110. Using network architecture 100, intermediate device 115 or a user of endpoint device 105 can now easily create the augmented environment described in this disclosure.

[0041] Modifications, additions, or omissions may be made to network architecture 100 without departing from the scope of the present disclosure. The present disclosure applies more generally to network architecture 100 including one or more endpoint devices 105, one or more wireless networks, one or more intermediate devices 115, one or more second networks 120, one or more relay servers 125, one or more third networks 130, and one or more endpoint manager servers 135, or any combination thereof.

[0042] Moreover, the separation of various components in the embodiments described herein does not indicate that the separation occurs in all embodiments. Furthermore, it will be understood, with the benefit of this disclosure, that the components described may be integrated together in a single component or separated into multiple components.

[0043] 2 shows an example environment 200 for augmenting reality by capturing signals using radios. The environment 200 may include a smart device 240 configured to wirelessly communicate with a computing device 250 and / or a sensor 255. The smart device 240, the computing device 250, and the sensor 255 may correspond to the intermediate device 115 and / or the endpoint device 105 of FIG. 1.

[0044] In some embodiments, smart device 240, computing device 250, and sensor 255 may be configured to communicate over network 245. Network 245 may include a public network (e.g., the Internet), a private network (e.g., a local area network (LAN) or a wide area network (WAN)), a wired network (e.g., an Ethernet network), a wireless network (e.g., an 802.xx network or a Wi-Fi network), a cellular network (e.g., a Long Term Evolution (LTE) or LTE-Advanced network, 1G, 2G, 3G, 4G, 5G, etc.), a router, a hub, a switch, a server computer, and / or combinations thereof. In some embodiments, smart device 240, computing device 250, and sensor 255 may be configured to communicate with each other over wireless network 110. In at least one embodiment, wireless network 110 and network 245 are the same network or include at least some overlapping components.

[0045] In some embodiments, sensor 255 may include a device configured to be physically attached to an object not configured for wireless communication (e.g., a non-computing device or a computing device that does not include a wireless card). For example, sensor 255 may be attached to a piece of furniture, a shelving unit, a photograph, a painting, a PC, or any other object. Sensor 255 may be configured to transmit a wireless signal that includes data corresponding to the object. For example, when sensor 255 is attached to furniture, sensor 255 may transmit a wireless signal that includes the dimensions, weight, and / or physical location of those pieces of furniture within environment 200. In at least one embodiment, sensor 255 may include a hardware secure element.

[0046] In some embodiments, computing device 250 may include a device configured for wireless communication. For example, computing device 250 may include a tablet, a smartphone, a mobile device, a smart television, a smart DVD player, or any other suitable device. Computing device 250 may transmit wireless signals that include data corresponding to computing device 250. For example, the wireless signals transmitted by computing device 250 may include a device identifier, a device descriptor, a physical location of the computing device, or any other suitable data.

[0047] Smart device 240 may receive wireless signals from computing device 250 and sensor 255. In some embodiments, smart device 240 may receive wireless signals via wireless network 110. In other embodiments, smart device 240 may receive wireless signals via network 245. In some embodiments, smart device 240 may receive a portion of the wireless signals via wireless network 110 and another portion of the wireless signals via network 245.

[0048] The smart device 240 may identify the computing device 250 and / or the sensor 255 based on the wireless signal. In some embodiments, the wireless signal may include metadata corresponding to the signal type of the wireless signal. For example, the metadata may indicate that the wireless signal is sent using a communication protocol, including Bluetooth, Wi-Fi, or another type of communication protocol. As another example, the metadata may indicate the frequency band of the wireless signal (e.g., the type of wireless signal). In these and other embodiments, the smart device 240 may identify the computing device 250 and / or the sensor 255 based on the metadata.

[0049] In some embodiments, the wireless signal may include data corresponding to the computing device 250 and / or the object to which the sensor 255 is attached. For example, the data included in the wireless signal may include a device identifier, a characteristic description, the issued MAC address of the computing device 250, or any other suitable data of the computing device 250 and / or the object to which the sensor 255 is attached. In these and other embodiments, the smart device 240 may identify the computing device 250 and / or the sensor 255 based on the data included in the wireless signal. For example, the smart device 240 may determine the issued MAC address of the computing device 250 to be included in the augmented environment.

[0050] In some embodiments, smart device 240 may determine that additional information about computing device 250 and / or object to which sensor 255 is attached may be available at relay server 125 and / or endpoint manager server 135 (not shown in FIG. 2 ). In these and other embodiments, smart device 240 may send a request message (e.g., a beacon) to relay server 125 or endpoint manager server 135 associated with one or more of computing device 250 and / or sensor 255. In these and other embodiments, the request message may identify the particular computing device 250 and / or sensor 255 from which the wireless signal was received. Additionally, the request message may request additional information about computing device 250 and / or sensor 255 from which the wireless signal was received.

[0051] The smart device 240 may determine the location of the computing device 250 and the sensor 255 within the environment 200. The location of the computing device 250 and the sensor 255 / object to which the sensor 255 is attached within the environment 200 may be determined based on the wireless signals. In some embodiments, the wireless signals may include location information of the corresponding computing device 250 and the sensor 255 / object to which the sensor 255 is attached. In other embodiments, the location of the corresponding computing device 250 and the sensor 255 / object to which the sensor 255 is attached may be determined based on characteristics of the wireless signals. For example, the signal strength of the wireless signals may be used.

[0052] In some embodiments, smart device 240 may determine the physical location of corresponding computing device 250 and sensor 255 / object to which sensor 255 is attached relative to each other. In these and other embodiments, smart device 240 may determine the physical location of corresponding computing device 250 and sensor 255 / object to which sensor 255 is attached relative to the physical location of smart device 240. Additionally or alternatively, smart device 240 may determine the physical location of corresponding computing device 250 and sensor 255 / object to which sensor 255 is attached relative to the external boundary of environment 200.

[0053] Smart device 240 may include a display 260. Smart device 240 may display the augmented environment via display 260. In some embodiments, the augmented environment may be displayed to represent environment 200 and different computing devices 250 and / or objects located within environment 200. For example, the augmented environment may include symbols representing smart device 240, one or more of computing devices 250, and / or one or more of the objects to which sensors 255 are attached. In some embodiments, smart device 240 may include a mobile device or a wearable device configured to move at least within environment 200. In other embodiments, smart device 240 may include a stationary device configured to remain stationary within environment 200.

[0054] The augmented environment may be displayed such that displayed symbols are displayed in locations within the augmented environment that correspond to the locations of corresponding computing devices 250 and objects within environment 200. In some embodiments, the augmented environment may include a three-dimensional representation of environment 200. For example, the augmented environment may include simulated walls of a room, heights of computing devices 250 or objects, shelves, or any other suitable devices and / or objects within environment 200. As another example, each piece of furniture in a room may be attached to a corresponding sensor 255. The corresponding sensor 255 may transmit wireless signals to include physical characteristics such as the furniture's dimensions, weight, shape, material, or manufacturer. Smart device 240 may receive these wireless signals and construct the augmented environment to be a three-dimensional representation and include a three-dimensional representation of each piece of furniture in the room.

[0055] An example implementation of the present disclosure may include a situation where environment 200 is dark (e.g., contains no light), but people moving around within environment 200 (e.g., military, police, first responders responding to an emergency or natural disaster, or other people) need to know what objects are within environment 200 so that they can move safely throughout environment 200. For example, a user may use smart device 240 to detect whether there are people trapped by rubble from a collapsed building after an earthquake. As another example, a user may use smart device 240 to detect whether there are people and / or animals trapped inside a building during an emergency.

[0056] In a particular example environment 200, the smart device 240 may generate an augmented environment representing a warehouse and assets within the warehouse (e.g., the assets may be objects with sensors 255 attached, for example, to the asset's packaging). A user of the smart device 240 may indicate that a particular asset is lost, such as by using a mobile application or website. For example, the lost status of the particular asset may be provided to the endpoint manager server 135 or to the relay server 125. In a first embodiment, the endpoint manager server 135 may send a message to the smart device 240 that a particular asset is lost. The message may include an identifier of the lost asset. The smart device 240 or a dongle communicatively coupled to the smart device 240 (e.g., a dongle attached to a personal computer configured to receive wireless signals so that the personal computer itself need not be within the warehouse) may be located within the warehouse. In some embodiments, the smart device 240 may only receive wireless signals from the sensors 255 because the assets are objects. In other embodiments, smart device 240 may receive wireless signals from both sensor 255 and computing device 250, but may filter out wireless signals received from computing device 250. Smart device 240 may filter out information corresponding to objects that are not the lost asset. Smart device 240 may display, via display 260, an augmented environment representing the warehouse. Additionally, smart device 240 may display a symbol representing the lost asset within the augmented environment. Additionally, in some embodiments, additional information about the lost asset may be displayed in the augmented environment. For example, the additional information may indicate the physical location of the lost asset within the warehouse.

[0057] A user may be able to determine the physical location or last known physical location of the lost asset within the warehouse based on the location of the symbol in the augmented environment and / or additional information. For example, the augmented environment may include simulated shelves that represent actual shelves within the warehouse, and a symbol representing the lost asset may be displayed as being on one of the simulated shelves. Thus, a user may determine which actual shelf corresponds to a simulated shelf in the augmented environment.

[0058] In another particular example, environment 200 may include a shopping area (e.g., a shopping mall, street shops, etc.). Computing device 250 and / or sensor 255 may be located within a store or attached to a storefront in the shopping area. A user of smart device 240 may enter the shopping area, and smart device 240 may receive wireless signals from computing device 250 and / or sensor 255. Smart device 240 may display an augmented environment representing the shopping area via display 260. Furthermore, smart device 240 may display symbols within the augmented environment representing each of the stores in the shopping area from which wireless signals were received from computing device 250 and sensor 255. Furthermore, in some embodiments, additional information about the stores may be displayed in the augmented environment. For example, the additional information may indicate current or upcoming sales, available goods or services, or current or upcoming events at the stores. Furthermore, in some embodiments, smart device 240 may compare goods or services included in a virtual wish list created by the user with the goods and services indicated in the wireless signals. In these and other embodiments, the augmented environment may include additional information indicating which products or services in the store are also included in the virtual wish list.

[0059] In yet another particular example, environment 200 may include a grocery store or another type of food vendor. Computing device 250 and / or sensor 255 may be located near or attached to food goods available in the grocery store. A user of smart device 240 may enter the grocery store, and smart device 240 may receive wireless signals from computing device 250 and / or sensor 255. Smart device 240 may display an augmented environment representing the grocery store via display 260. Additionally, smart device 240 may display one or more symbols within the augmented environment representing each of the food items for which wireless signals are received from computing device 250 or sensor 255. For example, computing device 250 may transmit wireless signals corresponding to each food item on a particular shelf or shelving unit. Additionally, in some embodiments, additional information about the food items may be displayed in the augmented environment. For example, the additional information may indicate at least one of the food's nutritional value, ingredients, health rating, gluten status (e.g., gluten-free or not), fat status (e.g., low-fat or not), and fiber status (e.g., high-fiber or not). Additionally, in some embodiments, smart device 240 may filter out symbols corresponding to computing devices 250 and / or sensors 255 that do not meet certain nutritional preferences indicated by the user. For example, symbols corresponding to foods that are high in fat or contain gluten may be filtered out and not displayed in the augmented environment.

[0060] In particular examples, the environment 200 may include a road system (e.g., surface road streets, highways, and / or interstates). One or more vehicles using the road system may include a computing device 250 and / or a sensor 255. Additionally, the smart device 240 may correspond to another vehicle (e.g., a police car, a highway patrol car, etc.). In some embodiments, a vehicle may not be complying with the law, for example, by not having proper insurance, not having current registration, or speeding. A user of the smart device 240 (e.g., a police officer) may decide to pull the vehicle over because it is not complying with the law. Once pulled over, the smart device 240 may receive one or more wireless signals from the computing device 250 in the vehicle or from a sensor 255 attached to the vehicle. The smart device 240 may display an augmented environment representing the road system and / or the vehicle via the display 260. Additionally, the smart device 240 may display additional information about the vehicle, the driver, the passengers, or some combination thereof in the augmented environment. For example, the additional information may indicate vehicle history, driver history (e.g., driving record or outstanding warrants), passenger history (e.g., outstanding warrants), or any other information corresponding to the vehicle, driver, or passenger.

[0061] In another particular example, environment 200 may include a medical facility (e.g., a particular room in a hospital, or an entire building). Computing device 250 and / or sensor 255 may be attached to a tool such as a scalpel, attached to a patient via the patient's clothing article (e.g., a wristband), or attached to an additional medical device. A user of smart device 240 may enter the medical facility, and smart device 240 may receive wireless signals from computing device 250 and / or sensor 255. In some embodiments, smart device 240 may display, via display 260, an augmented environment including symbols representing each of the tools and / or medical devices in the medical facility. Additionally, in some embodiments, the augmented environment may be displayed with a list of each of the tools and / or medical devices in the medical facility. In these and other embodiments, additional information including a list of each tool or medical device in environment 200 may enable each tool and / or medical device to be closely tracked during or after a procedure. Furthermore, the display of symbols in the augmented environment representing tools and medical devices may enable a user to confirm the location of each tool and medical device without causing problems. For example, a user may confirm that a scalpel is under the towel without having to move the towel. In these and other embodiments, the additional information may include information identifying the patient (e.g., identification information), the patient's medical history, medications taken by the patient, or any other suitable information.

[0062] In yet another particular example, the environment 200 may include an educational facility (e.g., a particular classroom or an entire building). The computing device 250 and / or the sensor 255 may correspond to the student's computing device or be attached to the student's wristband or backpack. For example, the computing device 250 may include a tablet, smartphone, laptop, or another computing device associated with the student. A user of the smart device 240 (e.g., a teacher, principal, assistant, or other school staff) may enter the educational facility, and the smart device 240 may receive wireless signals from the computing device 250 and / or the sensor 255. In some embodiments, the smart device 240 may display, via the display 260, an augmented environment including symbols representing each student who is present. In other embodiments, the augmented environment may be displayed to include only symbols for students who are not present. For example, the smart device 240 may include a list of all students enrolled in a class. The smart device 240 may receive the wireless signals and filter out each student who is present so that only symbols (e.g., avatars) representing students who are not present are displayed. In these and other embodiments, the additional information may include information identifying the student, the student's educational history, medical considerations about the student, or any other suitable information.

[0063] In another particular example, environment 200 may include a neighborhood, a city, a state, or other geographic environment. Computing device 250 and / or sensor 255 may be attached to objects and / or landmarks within environment 200. For example, sensor 255 may be attached to a park bench within a city. A user of smart device 240 may enter environment 200 (e.g., sit on a park bench), and smart device 240 may receive a wireless signal. In some embodiments, smart device 240, via display 260, may display an augmented environment including symbols representing landmarks around the object. For example, the augmented environment may display the location of a playhouse, viewing area, public attraction, or other landmark relative to smart device 240 within environment 200. Additionally, in some embodiments, the augmented environment may be displayed with a list of information describing the landmark, wait times for a show or line, the distance from the user's location to the landmark, or historical information about the landmark.

[0064] In yet another particular example, environment 200 may correspond to a building, structure, or other location open to the public. Computing device 250 and / or sensor 255 may correspond to or be attached to other people. A user of smart device 240 may enter environment 200, and smart device 240 may receive wireless signals from computing device 250 and / or sensor 255. In some embodiments, smart device 240 may include a list of the user's contacts or friends (e.g., friends on social media). Smart device 240 may filter out wireless signals corresponding to people not included in the list of contacts or friends. Additionally, smart device 240, via display 260, may display an augmented environment including symbols (e.g., avatars) representing each person who is within environment 200 and included in the user's list of contacts or friends.

[0065] Additionally, wireless signals transmitted by computing device 250 and sensor 255 may be encrypted to ensure secure transmission of data contained therein. The wireless signals may be encrypted according to any suitable encryption technique, such as a symmetric encryption algorithm (e.g., Advanced Encryption Standard (AES)) with a shared key that may be vendor-specific, user-specific, rotated or unrotated, or derived from shared information (e.g., the device's MAC address), or an asymmetric encryption algorithm such as, but not limited to, ECDSA.

[0066] Modifications, additions, or omissions may be made to environment 200 without departing from the scope of the present disclosure. The present disclosure applies more generally to an environment 200 that includes a smart device 240, one or more computing devices 250, one or more wireless networks 110, a network 245, one or more sensors 255, or any combination thereof. Furthermore, the present disclosure may also apply to an environment 200 that includes a smart device 240, one or more wireless networks 110, a network 245, one or more computing devices 250, and zero sensors 255. Furthermore, the present disclosure may also apply to an environment 200 that includes a smart device 240, one or more wireless networks 110, a network 245, one or more sensors 255, and zero computing devices 250.

[0067] Moreover, the separation of various components in the embodiments described herein does not indicate that the separation occurs in all embodiments. Furthermore, it will be understood, with the benefit of this disclosure, that the components described may be integrated together in a single component or separated into multiple components.

[0068] 3 is a flow diagram of an example method for augmenting reality by capturing signals using a radio. For example, method 300 may be implemented by smart device 240 of FIG. 2 to generate an augmented environment. Method 300 may be implemented to generate an augmented environment representing an environment that includes symbols representing computing devices and / or objects in the environment. For example, the method may be implemented to generate and display an augmented environment representing environment 200 that includes symbols representing computing device 250 and / or sensor 255 of FIG. 2.

[0069] The method may begin at block 305, where a wireless signal may be received from a device in an environment. For example, the wireless signal may be received by the smart device 240 from the computing device 250 and / or the sensor 255 of FIG.

[0070] At block 310, a device may be identified. In some embodiments, the device may be identified based on at least one of data and metadata included in the wireless signal. The data may include information identifying the device, the location of the device within the environment, the type of device, or any other suitable information. At block 315, a location of the device within the environment may be determined. The location of the device within the environment may be determined based on the data included in the wireless signal.

[0071] At block 320, an augmented environment may be displayed. The augmented environment may represent the environment. Additionally, a symbol representing the device may be displayed within the augmented environment. In some embodiments, the symbol may be displayed in a location within the augmented environment that corresponds to the location of the device within the environment. The augmented environment may include any type of data, such as images, icons, text, graphics, etc. The data may include information about the device, including information about how to purchase the device or similar devices and / or services related to the device.

[0072] 4 shows a diagrammatic representation of a machine in the exemplary form of a computing device 400 on which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies described herein. The computing device 400 may include a mobile phone, a smartphone, a netbook computer, a rack-mounted server, a router computer, a server computer, a personal computer, a mainframe computer, a laptop computer, a tablet computer, a desktop computer, etc. on which a set of instructions may be executed to cause the machine to perform any one or more of the methodologies described herein. In alternative embodiments, the machine may be connected (e.g., networked) to other machines in a LAN, an intranet, an extranet, or the Internet. The machine may operate in the capacity of a server machine in a client-server network environment. The machine may include a personal computer (PC), a set-top box (STB), a server, a network router, a switch, or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the machine. Furthermore, although only a single machine is shown, the term "machine" may also include any collection of machines that individually or together execute a set (or sets) of instructions to perform any one or more of the methodologies described herein.

[0073] The exemplary computing device 400 includes a processing device (e.g., a processor) 402, a main memory 404 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), a static memory 406 (e.g., flash memory, static random access memory (SRAM)), and a data storage device 416, which communicate with each other via a bus 408.

[0074] Processing device 402 represents one or more general-purpose processing devices, such as a microprocessor, a central processing unit, etc. More specifically, processing device 402 may include a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, or a processor implementing other instruction sets or combinations of instruction sets. Processing device 402 may also include one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a digital signal processor (DSP), a network processor, etc. Processing device 402 is configured to execute instructions 426 to perform the operations and steps described herein.

[0075] Computing device 400 may further include a network interface device 422 that may communicate with a network 418. Computing device 400 may also include a display device 410 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 412 (e.g., a keyboard), a cursor control device 414 (e.g., a mouse), and a signal generation device 420 (e.g., a speaker). In at least one embodiment, display device 410, alphanumeric input device 412, and cursor control device 414 may be combined into a single component or device (e.g., an LCD touchscreen).

[0076] The data storage device 416 may include a computer-readable storage medium 424 having stored thereon one or more sets of instructions 426 embodying any one or more of the methods or functions described herein. The instructions 426 may also reside completely or at least partially within the main memory 404 and / or within the processing device 402 during execution of the instructions 426 by the computing device 400, with the main memory 404 and the processing device 402 also constituting computer-readable media. The instructions may further be transmitted or received over the network 418 via the network interface device 422.

[0077] Although computer-readable storage medium 424 is shown to be a single medium in an exemplary embodiment, the term "computer-readable storage medium" may include a single medium or multiple media (e.g., centralized or distributed databases and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" may also include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a machine, causing the machine to perform any one or more of the methods of the present disclosure. The term "computer-readable storage medium" may therefore be taken to include, but is not limited to, solid-state memory, optical media, and magnetic media.

[0078] The present technology is illustrated, for example, according to various aspects described below. Various examples of aspects of the present technology are described as numbered examples (1, 2, 3, first, second, third, etc.) for convenience. These are provided as examples and do not limit the present technology. Aspects of various implementations described herein may be omitted, substituted with, or combined with aspects of other implementations, unless the context dictates otherwise. For example, one or more aspects of Example 1 below may be omitted, substituted with, or combined with one or more aspects of another example or examples (e.g., Example 2). The following is a non-limiting summary of some exemplary implementations presented herein.

[0079] A first example method includes receiving a wireless signal from a device within an environment, identifying the device based on at least one of data and metadata included in the wireless signal, determining a location of the device within the environment based on the data included in the wireless signal, and displaying an augmented environment representing the environment and a symbol representing the device within the augmented environment, wherein the symbol is displayed in a location within the augmented environment that corresponds to the location of the device within the environment.

[0080] The method of the first example, wherein the metadata included in the wireless signal includes a signal type for the wireless signal and a communication protocol for the wireless signal.

[0081] The method of the first example, wherein the data included in the wireless signal includes a device identifier.

[0082] The method of the first example, wherein the data contained in the wireless signal includes a characteristic description of the device.

[0083] The method of the first example, wherein the data contained in the wireless signal includes information related to purchasing the device or services related to the device.

[0084] The method according to the first example, wherein the method is performed by a mobile device configured to move at least within the environment.

[0085] The method of the first example, wherein displaying the augmented environment representing the environment includes constructing a three-dimensional representation of the environment in the augmented environment, including the device.

[0086] The method described in the first example, wherein the environment includes a shopping area, the device is incorporated into a storefront, and the data included in the wireless signal includes at least one of sales descriptions, product descriptions, and service descriptions available at a store corresponding to the storefront.

[0087] The method described in the first example, wherein the environment includes a shopping area, the device is incorporated into a shelf that stores food, and the data included in the wireless signal includes at least one of the nutritional value, ingredients, and health rating of the food.

[0088] The method described in the first example, wherein the environment includes a road system, the device is incorporated into a vehicle that uses the road system, and the data included in the wireless signal includes at least one of driver history, identification information, and vehicle identification information.

[0089] The method described in the first example, wherein the environment includes a medical facility, the device is incorporated into an article of clothing associated with a patient, and the data included in the wireless signal includes at least one of the patient's identification information, medical history, and medications taken by the patient.

[0090] The method described in a first example, wherein the environment includes a medical facility, the device is incorporated into a medical device, and a symbol representing the device is displayed in the augmented environment to track the medical device and ensure that the medical device is not misplaced.

[0091] The method described in the first example, wherein the environment includes an educational facility, the device is incorporated into an object associated with the student, and the method further includes marking the presence of the student based on a wireless signal.

[0092] The method of the first example, wherein the data contained in the wireless signal includes an issued MAC address of the device, and the method further includes determining the issued MAC address of the device.

[0093] The method of the first example, wherein the environment includes a residential building and the device includes a sensor physically attached to a non-computing device.

[0094] and displaying an augmented environment representing the environment and symbols representing the first device and the second device within the augmented environment, the symbols being displayed in locations within the augmented environment corresponding to the locations of the first device and the second device within the environment. A second example method includes: receiving a first wireless signal from a first device within the environment; receiving a second wireless signal from a second device within the environment; identifying the first device based on at least one of data and metadata included in the first wireless signal; identifying the second device based on at least one of data and metadata included in the second wireless signal; determining locations of the first device and the second device within the environment based on the first wireless signal and the second wireless signal; and displaying an augmented environment representing the environment and symbols representing the first device and the second device within the augmented environment, the symbols being displayed simultaneously in locations within the augmented environment corresponding to the locations of the first device and the second device within the environment.

[0095] The method described in the second example, wherein the data contained in the first wireless signal includes a characteristic description of the first device, and the first symbol representing the first device includes at least a portion of the characteristic description of the first device.

[0096] A third example method includes receiving a wireless signal from a device in an environment, identifying the device based on at least one of data and metadata included in the wireless signal, determining information of the device in the environment based on the data included in the wireless signal, and displaying an augmented reality element that indicates the information of the device, wherein the augmented reality element is displayed in a location within the augmented environment that corresponds to the location of the device within the augmented environment.

[0097] The method of the third example, wherein the augmented reality element includes a graphical representation of the device in an augmented environment.

[0098] The method of the third example, wherein the augmented reality element includes information about how to purchase the device or a service related to the device.

[0099] The terms used herein, and particularly in the appended claims (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" can be interpreted as "including, but not limited to," the term "having" can be interpreted as "having at least," the term "includes" can be interpreted as "includes, but is not limited to," etc.).

[0100] Furthermore, if a specific number of introduced claim recitations is intended, such intention will be expressly recited in the claim; absent such recitation, no such intention exists. For example, as an aid to understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases may not be interpreted as implying that introducing a claim recitation with the indefinite article "a" or "an" limits a particular claim containing such introduced claim recitation to embodiments containing only one such recitation (e.g., "a" and / or "an" may be interpreted to mean "at least one" or "one or more"); the same is true for the use of definite articles used to introduce claim recitations, even when the same claim includes the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" may be interpreted to mean "at least one" or "one or more").

[0101] Furthermore, even when a particular number of introduced claim recitations is explicitly recited, those skilled in the art will recognize that such recitation can be interpreted to mean at least the recited number (e.g., the bare recitation of "two recitations" without other modifiers means at least two recitations, or two or more recitations). Furthermore, in instances where language similar to "at least one of A, B, and C, etc." or "one or more of A, B, and C, etc." is used, such configurations are generally intended to include A only, B only, C only, A and B together, A and C together, B and C together, or A, B, and C together, etc. For example, use of the term "and / or" is intended to be construed in this manner.

[0102] Furthermore, any disjunction or phrase presenting two or more alternative terms, whether in the specification, claims, or drawings, may be understood to contemplate the possibility of including one of those terms, either of the terms, or both terms. For example, the phrase "A or B" may be understood to include the possibilities of "A" or "B" or "A and B."

[0103] The embodiments described herein may be implemented using computer-readable media for carrying or having computer-executable instructions or data structures stored thereon. Such computer-readable media may be any available media that can be accessed by a general-purpose or special-purpose computer. By way of example, and not limitation, such computer-readable media may include non-transitory computer-readable storage media including random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other optical disk storage, magnetic disk storage or other magnetic storage devices, flash memory devices (e.g., solid-state memory devices), or any other storage medium that can be used to carry or store desired program code in the form of computer-executable instructions or data structures and that can be accessed by a general-purpose or special-purpose computer. Combinations of the above may also be included within the scope of computer-readable media.

[0104] Computer-executable instructions may include, for example, instructions and data that cause a general-purpose computer, a special-purpose computer, or a special-purpose processing device (e.g., one or more processors) to perform a certain function or group of functions. Although the subject matter has been described in language specific to structural features and / or methodological acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms of implementing the claims.

[0105] As used herein, the term “module” or “component” may refer to a specific hardware implementation configured to perform the operations of a module or component and / or software object or routine that may be stored on and / or executed by general-purpose hardware (e.g., computer-readable media, processing device, etc.) of a computing system. In some embodiments, different components, modules, engines, and services described herein may be implemented as objects or processes (e.g., as separate threads) executing on a computing system. While some of the systems and methods described herein are generally described as being implemented in software (stored on and / or executed by general-purpose hardware), specific hardware implementations or combinations of software and specific hardware implementations are also possible and contemplated. As used herein, a “computing entity” may be any computing system as previously defined herein, or any module or any combination of modules running on a computing system.

[0106] For the disclosed processes and / or methods, the functions performed in the processes and methods may be implemented in different order as may be indicated by the context. Furthermore, the outlined steps and operations are provided only as examples, and some of the steps and operations may be optional, combined into fewer steps and operations, or expanded into additional steps and operations.

[0107] This disclosure may, at times, show different components contained within or connected to other different components. Such illustrated architectures are merely examples, and many other architectures may be implemented that achieve the same or similar functionality.

[0108] Aspects of the present disclosure may be embodied in other forms without departing from their spirit or essential characteristics. The described aspects are to be considered in all respects as illustrative and not restrictive. The claimed subject matter is indicated by the appended claims rather than by the foregoing description. All changes that come within the meaning and range of equivalency of the claims are to be embraced within their scope.

Claims

1. receiving wireless signals from devices in the environment; identifying the device based on at least one of data and metadata contained in the wireless signal; determining a location of the device relative to at least two other devices in the environment; displaying an augmented environment representing the environment and a symbol representing the device within the augmented environment, the symbol displayed in a location within the augmented environment corresponding to the location of the device within the environment; Including, comparing at least one of the data and metadata contained in the wireless signal with preset conditions, filtering out the wireless signal and displaying only symbols corresponding to the preset conditions; the environment includes a shopping area, the device is incorporated into a storefront, and the data included in the wireless signal includes at least one of sales descriptions, product descriptions, and service descriptions available at a store corresponding to the storefront; or the environment comprises a shopping area, the device is integrated into a shelf storing food products, and the data contained in the wireless signal comprises at least one of the nutritional value, ingredients, and health rating of the food products; or the environment includes a road system, the device is incorporated in a vehicle that uses the road system, and the data included in the wireless signal includes at least one of a driver history, an identification, and an identification of the vehicle; or the environment comprises a healthcare facility, the device is incorporated into an article of clothing associated with a patient, and the data contained in the wireless signal comprises at least one of an identification of the patient, a medical history, and medications taken by the patient; or the environment comprises a medical facility, the device is incorporated into a medical device, and the symbol representing the device is displayed in the augmented environment to track the medical device and ensure that the medical device is not misplaced; or The method, wherein the environment includes an educational facility, and the device is incorporated into an object associated with a student and includes marking the presence of the student based on the wireless signal.

2. The metadata included in the radio signal is a signal type for the wireless signal; and a communication protocol for the wireless signal; The method of claim 1 , comprising:

3. The method of claim 1 , wherein the data contained in the wireless signal includes a device identifier.

4. The method of claim 1 , wherein the data contained in the wireless signal includes a characteristic description of the device.

5. The method of claim 1 , wherein the data contained in the wireless signal includes information related to purchasing the device or services related to the device.

6. The method of claim 1 , wherein the method is performed by a mobile device configured to move at least within the environment.

7. The method of claim 1 , wherein displaying the augmented environment representing the environment comprises constructing a three-dimensional representation of the environment in the augmented environment that includes the device.

8. 10. The method of claim 1, wherein the data contained in the wireless signal includes an issued MAC address of the device, the method further comprising determining the issued MAC address of the device.

9. receiving a first wireless signal from a first device in the environment; receiving a second wireless signal from a second device in the environment; identifying the first device based on at least one of data and metadata included in the first wireless signal; identifying the second device based on at least one of data and metadata included in the second wireless signal; determining the relative locations of the first device and the second device within the environment; displaying an augmented environment representing the environment and symbols representing the first device and the second device within the augmented environment, the symbols simultaneously displayed in locations within the augmented environment corresponding to the locations of the first device and the second device within the environment; Including, comparing at least one of data and metadata contained in the first and second wireless signals with a set condition; filtering out the first and second wireless signals and displaying only symbols corresponding to the set condition; the environment includes a shopping area, the first device and the second device are incorporated into a storefront, and the data included in the wireless signal includes at least one of sales descriptions, product descriptions, and service descriptions available at a store corresponding to the storefront; or the environment comprises a shopping area, the first device and the second device are integrated into shelves storing food products, and the data included in the wireless signals comprises at least one of nutritional value, ingredients, and health ratings of the food products; or the environment includes a road system, the first device and the second device are incorporated in a vehicle that uses the road system, and the data included in the wireless signal includes at least one of a driver history, an identification, and an identification of the vehicle; or the environment comprises a healthcare facility, the first device and the second device are incorporated into an article of clothing associated with a patient, and the data contained in the wireless signal comprises at least one of an identification of the patient, a medical history, and medications taken by the patient; or the environment comprises a medical facility, the first device and the second device are incorporated into a medical device, and the symbol representing the device is displayed in the augmented environment to track the medical device and ensure that the medical device is not misplaced; or The method includes the environment including an educational facility, the first device and the second device being incorporated into an object associated with a student, and marking the presence of the student based on the wireless signal.

10. 10. The method of claim 9, wherein the data contained in the first wireless signal includes a characteristic description of the first device, and wherein a first symbol representing the first device includes at least a portion of the characteristic description of the first device.

Images