Control electronic devices and display information based on wireless distance measurement

Abstract

The present disclosure relates to controlling electronic devices and displaying information based on wireless ranging. A method for controlling an object by a wireless device, the method comprising: interpreting a first set of data to detect an intention gesture indicating an intention to control the object, wherein the object is within a wireless range of the wireless device; performing a wireless ranging operation to identify the object to be controlled; interpreting a second set of data to determine an action gesture, the action gesture indicating a command for controlling the object identified by the wireless ranging operation; and sending a command value corresponding to the command to control the object.

Description

[0001] This application is a divisional application of invention patent application No. 201780009457.0, filed on February 3, 2017, and entitled “Controlling electronic devices and displaying information based on wireless ranging”. Technical Field

[0002] Embodiments described herein relate to wireless communications between electronic devices and user interfaces, including systems and techniques for controlling electronic devices using gestures and measurements such as wireless ranging, and systems and techniques for displaying physical or augmented reality information on a user interface based at least in part on wireless ranging. Background Art

[0003] The use, and therefore popularity, of many electronic devices, including portable electronic devices such as cellular phones, is often influenced by ease of use. Specifically, the usability of many electronic devices is often determined by the user interface. The user interface is the means by which user actions or behaviors, including attempts by users to access features of the electronic device, are defined and received. Therefore, the user interface is an integral part of the overall user experience.

[0004] However, there are limitations associated with existing user interfaces, particularly user interfaces for small handheld electronic devices. For example, many existing user interfaces are limited by requiring a user to activate or turn on the electronic device. As a result, the user may need to make physical contact with the user interface, such as in the case of a user interface displayed on a touch-sensitive display. In addition, it may be difficult to present information on the user interface in a manner that allows different types of information to be intuitively distinguished from each other. Therefore, the amount of information that can be displayed on the user interface is often limited, and users are often confused by the information displayed. In addition to frustrating the user and degrading the user experience, there is often a significant opportunity cost associated with information that is ignored or not displayed. Therefore, many existing user interfaces produce frustrating user interactions, which degrades the user experience. Summary of the invention

[0005] Embodiments of wireless communication electronic devices related to controlling an object are disclosed herein. During operation, the wireless communication device identifies a received intention gesture indicating an intention to control an object, wherein the object is located in an environment close to the wireless communication device. The wireless communication device then utilizes wireless ranging to determine the object associated with the intention gesture. In addition, the wireless communication device accesses sensor data from one or more sensors associated with the wireless communication device and interprets the accessed sensor data to determine a command value, which is transmitted to control the object.

[0006] It should be noted that the object can be a physical object. For example, the physical object can include: a computing device, a display, a printer, a communication device, an audio device, an appliance, a wearable device, a home automation device, an environmental controller, and / or an accessory. Alternatively, the object can be a virtual object. In addition, the virtual object can act as a proxy to associate a command value with a physical object.

[0007] In addition, wireless ranging can determine the distance between the wireless communication device and the object using wireless transmission characteristics or wireless ranging calculations. In addition, the accessed sensor data may include an orientation value corresponding to the wireless communication device, which is determined using a compass, an accelerometer and / or a gyroscope.

[0008] Additionally, the command value may include a command for pairing the wireless communication device with the object; changing an operational setting of the object; and / or performing a function. Note that the command value may cause the object to transmit a response including a sensor value and / or may request access to a resource.

[0009] In some embodiments, a wireless communication device may utilize ultrasonic frequency modulation signals to determine the proximity of objects in the environment.

[0010] Also disclosed herein are embodiments of wireless communication electronic devices that are directed to locating an object. During operation, the wireless communication device receives a transmission having an identifier associated with an object that is located proximate to the wireless communication device in an environment. The wireless communication device determines a range of the object relative to the wireless communication device and / or a direction to the object, wherein the range, direction, or both are determined at least in part using wireless ranging. The wireless communication device then presents output information indicating the range and / or direction.

[0011] For example, presenting output information may involve displaying a map (or other representation) of a proximity area in an environment on a display in the wireless communication device and displaying a first indicator representing an object on the map. Alternatively or in addition, presenting output information may involve displaying an image of the proximity area in the environment captured by an imaging sensor in the wireless communication device on a display and displaying a second indicator representing an object on the image. In some embodiments, presenting output information involves transitioning between a map of the proximity area in the environment including a first indicator representing an object and an image captured by a camera including a second indicator corresponding to the object displayed on the display based on the orientation of the wireless communication device. It should be noted that the first indicator and / or the second indicator may be displayed regardless of whether the object is within the visual range of the wireless communication device or in other words is visible to the wireless communication device.

[0012] In addition, the identifier associated with the object can be a unique identifier. In addition, the identifier associated with the object can be associated with the resource class corresponding to the object. In some embodiments, multiple identifiers with different meanings or different resolvability can be associated with the same object.

[0013] In some embodiments, the wireless communication device compares the received identifier to one or more stored target identifiers to determine a match (e.g., to resolve the identifier). Note that the target identifier may have been received with the search result or transaction. Alternatively or in addition, the target identifier may correspond to a contact maintained by the wireless communication device.

[0014] In addition, the wireless communication device may transmit a message to the object based at least in part on the identifier. For example, the message may include an audio message that is transmitted as a beamformed audio message that is substantially perceptible only at a location corresponding to the object, wherein the location is determined at least in part based on a range and / or direction to the object.

[0015] Note that an object may be an electronic device associated with an individual. Alternatively or in addition, an object may be a resource in an environment, such as a public (eg, secure) resource or a private (eg, commercial) resource.

[0016] Additionally, the output information may include tactile output, which may be presented via the wireless communication device and / or another device.

[0017] Other embodiments provide a computer program product for use with a wireless communication device. The computer program product includes instructions for at least some of the operations performed by the wireless communication device.

[0018]

[0013] Additional embodiments provide methods that include one or more of the operations performed by a wireless communication device.

[0019] The purpose of providing this disclosure is to illustrate some exemplary embodiments to provide a basic understanding of some aspects of the subject matter described herein. Therefore, it should be understood that the features described above are only examples and should not be considered to narrow the scope or essence of the subject matter described herein in any way. Other features, aspects and advantages of the subject matter described herein will become apparent through the following detailed description, drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The included drawings are for illustrative purposes and are only used to provide examples of possible structures and arrangements for the disclosed systems and technologies, which are used to intelligently and efficiently manage operations, controls, interactions, and other communications between multiple associated devices, and for locating objects, such as through wireless ranging. These drawings in no way limit any changes in form and detail that may be made to the embodiments by those skilled in the art without departing from the spirit and scope of the embodiments. The embodiments will be readily understood by the detailed description below in conjunction with the accompanying drawings, in which like reference numerals represent similar structural elements.

[0021] Figure 1 is a block diagram illustrating an example of an electronic device that communicates wirelessly.

[0022] Figure 2 A block diagram is provided that illustrates an example of a wireless communication device that performs wireless ranging.

[0023] Figure 3 , Figure 4 and Figure 5 Diagrams are provided showing examples of controlling an object using one or more gestures based on wireless ranging.

[0024] Figure 6 An exemplary method for controlling an object utilizing a wireless communication device is provided.

[0025] Figure 7 supply Figure 1 An example of communication between electronic devices is shown.

[0026] Figure 8 Diagrams are provided showing examples of locating objects in an environment using wireless ranging.

[0027] Fig. 9 Provides display maps and representations Figure 8 An illustration of an example of a user interface with an indicator for an object in FIG.

[0028] Fig.10 Provides images and representations showing Figure 8 An illustration of an example of a user interface with an indicator for an object in FIG.

[0029] Fig.11 Diagrams are provided showing examples of different modes of a user interface based on the orientation of a wireless communication device.

[0030] Fig.12 Diagrams are provided showing examples of different modes of a user interface.

[0031] Fig.13 Diagrams showing examples of augmented reality are provided.

[0032] Fig.14 Diagrams are provided showing examples of extended situational awareness regarding resources in an environment.

[0033] Fig.15 Diagrams are provided showing examples of providing reminders and tracking objects in an environment.

[0034] Fig.16 Diagrams are provided showing examples of different types of feedback associated with user interfaces.

[0035] Fig.17 An example of a method for locating an object using a wireless communication device is provided.

[0036] Fig.18 supply Figure 1 Another example of communication between electronic devices is shown.

[0037] Note that similar reference numerals refer to corresponding parts throughout the drawings. In addition, multiple instances of the same component are designated by a common prefix that is separated from the instance number by a dash. DETAILED DESCRIPTION

[0038] The disclosed embodiments relate to wireless communication devices (such as smart phones or smart watches) that use one or more measurements (such as wireless ranging or radio-based distance measurements) to remotely control objects such as physical devices or virtual representations (or placeholders) associated with physical devices at physical reference locations. In particular, the wireless communication device can implement user interface techniques in which one or more measurements are used to remotely (e.g., from a distance, and generally, without physical contact) control the operation of an object. Note that objects are sometimes referred to as "controllable devices" or "target devices."

[0039] The user interface technology may eliminate constraints associated with many existing user interfaces. For example, in order to control an object, a user may no longer need to turn on or unlock a wireless communication device (e.g., by providing a password or a biometric identifier such as a fingerprint). Similarly, by facilitating control of an object from a distance, the user interface technology may eliminate the need for a user to physically contact a user interface on or associated with the object. Thus, the user interface technology may improve the user experience when using a wireless communication device and an object, thereby increasing customer satisfaction and retention.

[0040] The disclosed embodiments also relate to a wireless communication device (such as a smart phone or smart watch) that uses one or more measurements (such as wireless ranging or radio-based distance measurement) to locate a proximity object in an environment, such as a physical device (e.g., an electronic device or resource) or a virtual representation (or placeholder) of a physical device at a physical reference location. The virtual representation can be a physical object (e.g., a placeholder object) or a completely virtual object (e.g., one or more coordinates corresponding to a location). The wireless communication device may implement a user interface technique in which an identifier associated with an object is received wirelessly, and one or more measurements are used to wirelessly (e.g., from a distance, and generally, without physical contact) determine the range of the object relative to the wireless communication device and / or the direction to the object (e.g., a vector or heading). The wireless communication device may then present output information indicating the range and / or direction. For example, the wireless communication device may display a map of a proximity area in the environment with an indicator on the map representing the object. Alternatively, the wireless communication device may display an image of the proximity area (e.g., captured by a camera of the wireless communication device) with an indicator on the image representing the object.

[0041] This additional user interface technology can eliminate the constraints associated with many existing user interfaces. For example, output information can be displayed according to range and / or direction. This can allow the displayed output information to be intuitively arranged, which can allow the user to quickly understand and use what may otherwise be complex content. In addition, this additional user interface technology can allow output information to be displayed only when it is useful to the user, so that only a relevant subset of information of interest is displayed at any given time. Specifically, the relevant subset can be identified based on the location of the proximity object in the surrounding environment and the situational awareness of the user's behavior (such as the orientation of the wireless communication device or the direction it is pointed). Therefore, the additional user interface technology can improve the user experience when using the wireless communication device, thereby improving customer satisfaction and retention.

[0042] It is noted that the communication used during wireless communication between electronic devices in the user interface technology can comply with a communication protocol, such as the Institute of Electrical and Electronics Engineers (IEEE) 802.11 standard (sometimes referred to as Wi-Fi). For example, the communication can be used with IEEE 802.11ax, which is used as an illustrative example in the following discussion. However, the user interface technology can also be used with a variety of other communication protocols and can also be used in electronic devices (such as portable electronic devices or mobile devices) that can combine a variety of different radio access technologies (RATs) to provide connections through different wireless networks that provide different services and / or capabilities.

[0043] Specifically, the wireless communication device may include a device for supporting communication according to a wireless personal area network (WPAN) communication protocol (such as The wireless communication device may communicate via a wireless wide area network (WWAN), a wireless metropolitan area network (WMAN), a WLAN, a near field communication (NFC), a cellular telephone or a data network (such as using a third generation (3G) communication protocol, a fourth generation (4G) communication protocol such as long term evolution or LTE, advanced LTE (LTE-A), a fifth generation (5G) communication protocol, or other currently or future developed advanced cellular communication protocols), and / or another communication protocol.

[0044] In some embodiments, the wireless communication device may also operate as part of a wireless communication system, which may include a set of client devices, which may also be referred to as stations, client electronic devices, or client electronic devices, interconnected to access points (e.g., as part of a WLAN) and / or to each other (e.g., as part of a WPAN and / or an "ad hoc" wireless network such as Wi-Fi Direct). In some embodiments, a client device may be any electronic device capable of communicating via WLAN technology (e.g., according to a WLAN communication protocol). In addition, in some embodiments, the WLAN technology may include a Wi-Fi (or more generally, a WLAN) wireless communication subsystem or radio component, and the Wi-Fi radio component may implement the Institute of Electrical and Electronics Engineers (IEEE) 802.11 technology, such as one or more of the following: IEEE 802.11a; IEEE 802.11b; IEEE 802.11g; IEEE 802.11-2007; IEEE 802.11n; IEEE 802.11-2012; IEEE 802.11ac; IEEE 802.11ax, or other IEEE 802.11 technologies developed now or in the future.

[0045] Thus, in some embodiments, a wireless communication device may act as a communication hub that provides access to a WLAN and / or to a WWAN, and thus provides access to a variety of services that may be supported by various applications executing on the wireless communication device. Thus, a wireless communication device may include an "access point" that communicates wirelessly with other electronic devices (such as using Wi-Fi) and provides access to another network (such as the Internet) via IEEE 802.3 (sometimes referred to as "Ethernet").

[0046] In addition, it should be understood that the electronic devices described herein may be configured as multimode wireless communication devices that can also communicate via different 3G and / or second generation (2G) RATs. In these scenarios, the multimode electronic device or UE may be configured to prefer attaching to an LTE network that provides a faster data rate throughput than other 3G traditional networks that provide a lower data rate throughput. For example, in some specific implementations, the multimode electronic device is configured to fall back to a 3G traditional network, such as an evolved high speed packet access (HSPA+) network or a code division multiple access (CDMA) 2000 evolution-data only (EV-DO) network when LTE and LTE-A networks are not available.

[0047] Wireless ranging can be performed using any standard or proprietary ranging technology, or any combination of standard and / or proprietary ranging technologies. Wireless ranging operations can be performed to determine the distance between devices (e.g., between an initiator and a responder), the direction between devices, or both. For example, a flight time / arrival time (ToF / ToA) can be determined for one or more messages between devices, which can be used to establish a distance measurement. The one or more messages can have any format and can be transmitted using any wireless protocol such as 802.11 protocol, Bluetooth, etc. In some embodiments, ToF / ToA can be determined using a two-way exchange of two or more messages. In addition, in some embodiments, one or more messages used to perform ranging can be protected, for example, by encrypting or otherwise protecting at least a portion of the content. In addition, in some embodiments, the direction of the source of one or more wireless signals can be determined using a technique such as angle of arrival (AoA). For example, AoA estimation can be performed using multiple receiving elements (e.g., elements of an antenna array) to measure different times of arrival (TDOA) and / or different phases (PDOA) of signals. Additionally or alternatively, in some embodiments, directivity can be determined by measuring Doppler shift to establish frequency difference of arrival (FDOA). Wireless ranging techniques can be applied alone or in combination to perform a single ranging operation. In addition, wireless ranging techniques can be applied alone or in combination to perform continuous ranging operations, such as continuous or discontinuous ranging, and the history of measurements can be captured and used in performing operations based on range and / or direction.

[0048] In accordance with various embodiments described herein, the terms "wireless communication device," "electronic device," "mobile device," "mobile station," "wireless station," "wireless access point," "station," "access point," and "user equipment (UE)" may be used herein to describe one or more consumer electronic devices that may be capable of performing processes associated with various embodiments of the present disclosure.

[0049] An implementation scheme of the user interface technique is now described. Figure 1A block diagram 100 is provided showing an example of an electronic device that communicates wirelessly. Specifically, a wireless communication device 110 (such as a smart phone, laptop, wearable device, or tablet) and a physical device 112 can communicate wirelessly. These electronic devices can communicate wirelessly in the following situations: detect each other by scanning a wireless channel, transmit and receive beacons or beacon frames on a wireless channel, establish a connection (e.g., by transmitting a connection request), and / or transmit and receive packets or frames (which may include requests and / or additional information such as data as a payload). In addition, a virtual representation 114 may exist at a physical location 116. The virtual representation 114 may correspond to a physical device 118, so that the virtual representation 114 acts as a proxy for the physical device 118. In this case, the wireless communication device 110 can identify and interact with the virtual representation 114, but can transmit wireless signals received at the physical device 118. For example, the virtual representation 114 may be associated with a thermostat, and adjustments to the thermostat by the wireless communication device 110 may be provided through interaction with the virtual representation 114, but received and implemented by an environmental unit (eg, the physical device 118).

[0050] It is noted that physical devices 112 and / or 118 may include: an appliance (such as an oven, toaster, refrigerator, dishwasher, or washing machine), another electronic device (such as a computer, laptop, tablet, or computing device), an entertainment device (such as a television, display, radio receiver, or set-top box), an audio device, a projector, a security device (such as an alarm or door lock), a communication device (such as a smart phone), a surveillance device (such as a smoke detector or a carbon monoxide detector), an environmental controller (such as a thermostat, a light switch, or a curtain), an accessory (such as a keyboard, a mouse, or a speaker), a printer, a wearable device, a home automation device, a resource in environment 108 (such as a transportation resource, a shared computing resource, a medical resource, a display resource, a security resource, a reachability resource, or a safety resource), etc. In addition, virtual representation 114 may be implemented as: a sticker, a picture, a ceramic piece, a geo-fence, one or more coordinates defining a location, etc. In some embodiments, physical device 118 includes: a light switch, a thermostat, etc.

[0051] As referenced below Figure 2As further described, the wireless communication device 110, the physical device 112, and / or the physical device 118 may include subsystems, such as a networking subsystem, a memory subsystem, and a processor subsystem. In addition, the wireless communication device 110, the physical device 112, and / or the physical device 118 may include a radio component 120 in the networking subsystem. More generally, the wireless communication device 110, the physical device 112, and / or the physical device 118 may include any electronic device (or may be included in any electronic device) having a networking subsystem that enables the wireless communication device 110, the physical device 112, and / or the physical device 118 to wirelessly communicate with another electronic device. This may include transmitting beacons on a wireless channel to enable electronic devices to initially contact or detect each other, and then exchanging subsequent data / management frames (such as connection requests) to establish a connection (sometimes referred to as a "Wi-Fi connection"), configuring security options (e.g., IPSec), transmitting and receiving packets or frames via the connection, etc.

[0052] like Figure 1 As can be seen in the figure, wireless signals 122 (represented by jagged lines) are communicated by radio components 120-1 and 120-2 in wireless communication device 110 and physical device 112, respectively. For example, wireless communication device 110 and physical device 112 may exchange packets using the Bluetooth protocol in a wireless personal area network (WPAN) or the Wi-Fi protocol in a wireless local area network (WLAN).

[0053] Specifically, as shown below Figure 3-Figure 5 and Figure 8-Figure 16As described, the wireless communication device 110 may transmit a frame or packet including a transmission time. When this frame or packet is received by the physical device 112, the arrival time may be determined. Based on the product of the flight time (the difference between the arrival time and the transmission time) and the propagation speed, the distance between the wireless communication device 110 and the physical device 112 may be calculated. The distance may be transmitted from the physical device 112 to the wireless communication device 110 together with an identifier (such as a unique identifier) ​​of the physical device 112 or the user of the physical device 112 in a subsequent frame or packet transmission. Alternatively, the physical device 112 may transmit a frame or packet including the transmission time and the identifier of the physical device 112, and the wireless communication device 110 may determine the distance between the wireless communication device 110 and the physical device 112 based on the product of the flight time (the difference between the arrival time and the transmission time) and the propagation speed. It should be noted that this method for dynamically determining the distance between electronic devices for wireless communication is sometimes referred to as "wireless ranging." Additionally, when multiple target devices may be positioned close to one another or in the same line of sight, wireless ranging (alone or in conjunction with other sensor inputs such as a compass, gyroscope, and / or accelerometer) may be used to disambiguate control input intent. A variation of this approach may be used in which the wireless communication device 110 senses gesture input directed to a physical location 116 via sensor inputs (e.g., a compass, gyroscope, and / or accelerometer) and determines that one or more control signals should be transmitted to an associated device, such as a physical device 118 associated with a virtual representation 114. Similarly, another variation of this approach in which the wireless communication device 110 transmits a frame or packet that is reflected at the physical location 116 may optionally be used to dynamically determine the distance between the wireless communication device 110 and the virtual representation 114. Thus, wireless ranging may be used by the wireless communication device 110 to determine when an object (such as a physical device 112 or a virtual representation 114) is in proximity in the environment 108.

[0054] While the foregoing examples illustrate wireless ranging with synchronized clocks in the wireless communication device 110 and the physical device 112, in other embodiments, the clocks are not synchronized. For example, the location of the wireless communication device 110 or the physical device 112 may be estimated based on propagation velocity and arrival time data (sometimes referred to as "differential time of arrival") of the wireless signal 122 at several receivers at different known locations, even when the transmission time is unknown or unavailable. More generally, a variety of radio positioning techniques may be used, such as: determining distance based on the power difference of the received signal strength indicator (RSSI) relative to the original transmission signal strength (which may include corrections for absorption, refraction, shadowing, and / or reflection); determining the angle of arrival at a receiver using a directional antenna or based on the differential time of arrival at an antenna array with known locations (including non-line-of-sight reception); determining distance based on backscattered wireless signals; and / or determining the angle of arrival at two receivers with known locations (i.e., trilateration or multilateration). Note that wireless signal 122 may include transmissions over a GHz or multi-GHz bandwidth to produce pulses of short duration (such as, for example, approximately 1 ns), which may allow distance to be determined within 0.3 m (e.g., 1 foot). In some embodiments, wireless ranging utilizes location information such as that determined or specified by a local positioning system, a global positioning system, and / or a wireless network. Figure 1 is facilitated by the location of one or more of the electronic devices.

[0055] In addition, the wireless communication device 110 may include one or more sensors that measure (or generate) sensor data. For example, the one or more sensors may include: one or more compasses, one or more accelerometers, and / or one or more gyroscopes that measure the orientation (or orientation value) or direction of the wireless communication device 110; one or more accelerometers that measure the acceleration of the wireless communication device 110; a transceiver (such as a radio component 120-1) that determines a measure of wireless communication between the wireless communication device 110 and another electronic device (such as a physical device 112, or in an embodiment where a reflected wireless signal is received, the wireless communication device 110); one or more touch sensors configured to receive touch input, for example, via a touch screen; and / or one or more microphones or acoustic transducers that measure ambient sound in the environment 108 of the wireless communication device 110. In some embodiments, the wireless communication device 110 uses an ultrasonic chirp signal (chirp) provided by an acoustic transducer to determine the proximity of an object in the environment 108. This ultrasonic chirp signal may be outside the range of human hearing. In the following discussion, "proximity" of an electronic device should be understood to include at least within a wireless communication range, and the electronic device may be further limited to being in the same room or within a predetermined distance (such as within 10 meters or 30 meters).

[0056] As referenced below Figure 3-Figure 5 As further described, during the user interface technique, the wireless communication device 110 may use wireless ranging and / or sensor data to control the physical devices 112 and / or 118. Specifically, the wireless communication device 110 may identify an intended gesture by a user of the wireless communication device 110. For example, the user may point the wireless communication device 110 toward the physical location 116 (and therefore toward the virtual representation 114) or toward the physical device 112, and the wireless communication device 110 may identify this "intended gesture" based on the orientation of the wireless communication device 110 (in isolation or in relation to the location and / or orientation of the physical device 112 or virtual representation 114) and / or the movement of the wireless communication device 110 (e.g., a predefined intended gesture motion, such as a forward reach (or push)). The wireless communication device 110 may then determine an object (such as the physical device 112 or virtual representation 114) associated with the intended gesture, including utilizing wireless ranging. For example, the object corresponding to the intended gesture may be determined based at least in part on the distance to the object and the orientation (or relative orientation) determined utilizing wireless ranging.

[0057] In addition, the user may then perform a "motion gesture" by, for example, moving the wireless communication device 110 in two or three dimensions. The motion gesture may be captured in detected / measured sensor data acquired by one or more sensors in or associated with the wireless communication device 110. The wireless communication device 110 may then interpret the sensor data to determine the motion gesture, and therefore determine a command or command value. In addition, the wireless communication device 110 may transmit the command or command value in a frame or packet to a corresponding device to be controlled, such as the physical device 112 or the physical device 118.

[0058] As further described below, it is noted that command values ​​may include commands for pairing or associating the wireless communication device 110 with an object; changing an operational setting of an object (e.g., turning a light on or off, or changing a temperature on a thermostat, adjusting volume, channel, and / or playback settings); transmitting data (such as media or text data); and / or performing any other such function. It is noted that command values ​​may cause the object to transmit a response (e.g., including a sensor value) back to the wireless communication device 110 and / or may request access to a resource (such as a wireless printer).

[0059] Thus, the user interface techniques may allow the wireless communication device 110 to control an object (or functionality associated with an object) from a distance, including without the user turning on or unlocking the wireless communication device 110. This capability may also provide the user with new degrees of freedom in controlling or interacting with the object. Thus, the user interface techniques may improve the user experience when using the wireless communication device 110 and objects, thereby increasing user satisfaction and retention.

[0060] In the described embodiment, processing a packet or frame in one of the wireless communication device 110, the physical device 112, and / or the physical device 118 includes: receiving a wireless signal 122 that encodes the packet or frame; decoding / extracting the packet or frame from the received wireless signal 122 to obtain the packet or frame; and processing the packet or frame to determine information contained in the packet or frame (such as data in a payload).

[0061] In general, communication via WLAN in user interface technology can be characterized by a variety of metrics (or communication performance metrics). For example, the metrics may include: RSSI, data rate, data rate of successful communication (sometimes referred to as "throughput"), error rate (such as retry or retransmission rate), mean square error of equalized signal relative to equalization target, inter-symbol interference, multipath interference, signal-to-noise ratio, width of eye diagram, ratio of number of bytes successfully communicated during time interval (such as 1 second-10 seconds) to the estimated maximum number of bytes that can be communicated in the time interval (where the latter is sometimes referred to as the "capacity" of the communication channel or link), and / or ratio of actual data rate to estimated data rate (sometimes referred to as "usage").

[0062] As referenced below Figure 8-Figure 16 As further described, during the user interface technique, the wireless communication device 110 may use wireless ranging and / or sensor data to locate an object in the environment 108 and provide intuitive information about the object based on the determined location. Specifically, the wireless communication device 110 may receive a transmission that includes an identifier associated with an object (e.g., a physical device 112) located proximate to the wireless communication device 110 in the environment 108 (such as a room in a building or an area around the wireless communication device 110). For example, the identifier may be a unique identifier of the physical device 112 (and therefore, a unique identifier of the user of the physical device 112). Alternatively or in addition, the identifier may be associated with a resource class corresponding to the physical device 112, such as: transportation resources, shared computing resources (e.g., printers), medical resources, display resources, security resources, accessibility resources, safety resources, etc. It is noted that the identifier may have been received with the search results or transactions. Alternatively or in addition, the identifier may correspond to a contact of the user of the wireless communication device 110. In some implementations, an object (eg, physical device 112) can separately transmit multiple different identifiers that can be resolved differently based on information available to a recipient (eg, wireless communication device 110).

[0063] In response, the wireless communication device 110 may compare the received identifier to one or more stored target identifiers to determine a match. Thus, using wireless ranging, the wireless communication device 110 may determine the range (or distance) of the physical device 112 relative to the wireless communication device 110 and / or the direction to the physical device 112.

[0064] The wireless communication device 110 may then present output information indicating the range and / or direction to the physical device 112. For example, as described below with reference to Fig. 9 As further described, the wireless communication device 110 may display a map of the proximity area in the environment 108 and may display a first indicator representing the physical device 112 on the map. Alternatively or additionally, as described below with reference to Fig.10 Further described, the wireless communication device 110 may display an image of the proximity area in the environment 108 captured by an imaging sensor (such as a CMOS or CCD imaging sensor, and more generally, a camera) in the wireless communication device 110, and may display a second indicator (which may be the same as or different from the first indicator) representing the physical device 112 on the image. It should be noted that the first indicator and / or the second indicator may be displayed regardless of whether the physical device 112 is within the visual range of the wireless communication device 110 or is otherwise visible to the wireless communication device 110.

[0065] In addition, if Fig.11 As depicted in , in some embodiments, a user can intuitively switch back and forth or transform between the two user interface modes by changing the orientation of the wireless communication device 110. Specifically, based on the measured orientation value, the wireless communication device 110 can selectively transition between displaying a map and displaying an image. Thus, when the wireless communication device 110 is facing downward (e.g., horizontally), a map with a first indicator may be displayed, and when the wireless communication device 110 is facing the direction of the physical device 112 (e.g., vertically), an image with a second indicator may be displayed.

[0066] Alternatively, if Fig.12, a user may select a user interface mode, such as by making and then breaking contact with a touch-sensitive display in the wireless communication device 110 within a tap area of ​​an icon. In this way, a user may select a user interface mode in which a map with a 360° view around the wireless communication device 110 is displayed, and objects (such as the physical device 112) at different ranges and directions are represented on the map by a first indicator (such as a symbol, emoticon, or graphical icon). In addition, when the wireless communication device 110 is pointed at the physical device 112 or facing the direction of the physical device 112, the user may select a user interface mode in which an image is displayed and the physical device 112 is represented by a second indicator (such as a symbol, emoticon, or graphical icon). In some embodiments, one or more other graphical elements (e.g., directional elements) may also be presented when the display of the wireless communication device 110 displays an image of the proximity area.

[0067] In some embodiments, the user interface technology includes augmented reality. Specifically, as described below with reference to Fig.13 As described, a forward-facing camera on the wireless communication device 110 may capture an image (or video) of the environment 108 that would be obscured by the wireless communication device 110 when the user holds the wireless communication device 110 in front of their face. Thus, when the wireless communication device 110 is facing in the direction of an object (such as the physical device 112), the image may be displayed on the wireless communication device 110 along with an indicator representing the physical device 112, such that the wireless communication device 110 appears transparent to the user. Note that the indicator may be positioned in the image at its location in the environment 108 and may be sized to provide a perspective view with intuitive information about the relative range to the object. Thus, objects that are farther away from the wireless communication device 110 (and therefore from the user) may be represented by smaller indicators than objects that are closer to the wireless communication device 110.

[0068] In addition, as referenced below Fig.14As described, augmented reality may include so-called "X-ray vision." Specifically, when the physical device 112 is located on the other side of an opaque barrier (such as a wall or a piece of furniture), the image displayed on the wireless communication device 110 may allow the user to see the physical device 112 as if the opaque barrier did not exist, or as if the user could see through the opaque barrier. For example, if the user is looking for a network printer located in another room, when the user points the wireless communication device 110 in the direction of the network printer, an image of the network printer may be displayed on the wireless communication device 110 along with information allowing the user to access the network printer. Thus, by tapping an icon associated with the network printer in a user interface displayed on the wireless communication device 110, the user may instruct the wireless communication device 110 to pair or connect to (or otherwise interact with) the network printer. Alternatively or in addition, the user interface may include instructions on how to reach the network printer (and optionally also a building map).

[0069] Similarly, as referenced below Fig.15 As further described, the user interface technology may allow a user to track the location of an object (such as physical device 112) in environment 108, or to obtain reminders (e.g., to prevent loss) about one or more related objects in environment 108. In this way, a parent may keep track of a child, or may ensure that something (such as a backpack) is not forgotten on the way to school.

[0070] In addition, as referenced below Fig.16 As further described, a variety of feedback may be provided to a user or other individuals in the environment 108 using user interface techniques. In addition to the visual information previously described, the wireless communication device 110 may provide tactile feedback to the user directly on the wireless communication device 110 or indirectly through another electronic device (such as the user's smart watch). In this way, the user may receive feedback about his environment, such as when he is facing the direction of the physical device 112.

[0071] In addition, the wireless communication device 110 may transmit a message, such as a text or audio message, to the physical device 112. For example, the message may include an audio message (e.g., a private audio broadcast) that is transmitted as a beamformed audio message that is substantially perceptible only at a location corresponding to the physical device 112. In this way, a user of the wireless communication device 110 may send a private message to another individual within range, even when the user does not have an electronic means for communicating with the other individual (such as when the user does not know the recipient's phone number or email address). In addition or alternatively, the wireless communication device 110 may transmit a beamformed audio message to the user with relevant information, such as: the name of a person in the environment 108, a task to be completed, an upcoming appointment, etc.

[0072] Thus, the user interface techniques may allow the wireless communication device 110 to control an object (or functionality associated with an object) from a distance, including without the user turning on or unlocking the wireless communication device 110. The user interface techniques may also allow the wireless communication device 110 to provide situational awareness about objects in the environment 108 in an intuitive and useful manner. These capabilities may also provide the user with new degrees of freedom in how they communicate and interact with objects in the environment 108. Thus, the user interface techniques may improve the user experience when using the wireless communication device 110, and thus may increase user satisfaction and retention.

[0073] In the described embodiment, processing a packet or frame in one of the wireless communication device 110, the physical device 112, and / or the physical device 118 includes: receiving a wireless signal 122 that encodes the packet or frame; decoding / extracting the packet or frame from the received wireless signal 122 to obtain the packet or frame; and processing the packet or frame to determine information contained in the packet or frame (such as data in a payload).

[0074] Typically, communication via WLAN in user interface technology can be characterized by a variety of metrics (or communication performance metrics). For example, the metrics may include: RSSI, data rate, data rate of successful communications (sometimes referred to as "throughput"), error rate (such as retry or retransmission rate), mean square error of the equalized signal relative to the equalization target, inter-symbol interference, multipath interference, signal-to-noise ratio, width of the eye diagram, ratio of the number of bytes successfully communicated during a time interval (such as 1 to 10 seconds) to the estimated maximum number of bytes that can be communicated in the time interval (where the latter is sometimes referred to as the "capacity" of the communication channel or link), and / or ratio of the actual data rate to the estimated data rate (sometimes referred to as "usage").

[0075] Although we Figure 1 The network environment shown is described as an example, but in alternative embodiments, there may be different numbers or types of electronic devices. For example, some embodiments include more or fewer electronic devices. For example, in another embodiment, different electronic devices may be transmitting and / or receiving packets or frames.

[0076] Embodiments of electronic devices are now described. Figure 2 A block diagram showing an example of an electronic device 200 (which may be a portable electronic device or a station). For example, the electronic device 200 may be one of the following: Figure 1The wireless communication device 110, the physical device 112, and / or the physical device 118 in the electronic device 200. The electronic device 200 may include a processing subsystem 210, a memory subsystem 212, a networking subsystem 214, a display subsystem 226, a measurement subsystem 230, and a user interaction subsystem 232. The processing subsystem 210 includes one or more devices configured to perform computing operations. For example, the processing subsystem 210 may include one or more microprocessors, application specific integrated circuits (ASICs), microcontrollers, programmable logic devices, and / or one or more digital signal processors (DSPs).

[0077] The memory subsystem 212 includes one or more devices for storing data and / or instructions for the processing subsystem 210 and the networking subsystem 214. For example, the memory subsystem 212 may include dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory, and / or other types of memory. In some embodiments, the instructions for the processing subsystem 210 in the memory subsystem 212 include: one or more program modules or instruction sets (such as program modules 222 or operating system 224) that can be executed by the processing subsystem 210. For example, ROM can store programs, utilities, or processes to be executed in a non-volatile manner, and DRAM can provide volatile data storage and can store instructions related to the operation of the electronic device 200. It should be noted that the one or more computer programs may constitute a computer program mechanism or software. In addition, the instructions in the various modules in the memory subsystem 212 can be implemented in the following languages: high-level procedural languages, object-oriented programming languages, and / or assembly languages ​​or machine languages. Furthermore, the programming language may be compiled or interpreted, e.g., configurable or configured (the two are used interchangeably in this discussion) to be executed by the processing subsystem 210. In some embodiments, the one or more computer programs are distributed over a network-coupled computer system so that the one or more computer programs are stored and executed in a distributed manner.

[0078] In addition, the memory subsystem 212 may include mechanisms for controlling access to memory. In some embodiments, the memory subsystem 212 includes a memory hierarchy that includes one or more caches coupled to memory in the electronic device 200. In some of these embodiments, one or more of the caches are located in the processing subsystem 210.

[0079] In some embodiments, the memory subsystem 212 is coupled to one or more high-capacity mass storage devices (not shown). For example, the memory subsystem 212 may be coupled to a magnetic disk drive or optical disk drive, a solid-state drive, or another type of mass storage device. In these embodiments, the memory subsystem 212 may be used by the electronic device 200 as a fast access storage device for frequently used data, while the mass storage device is used to store less frequently used data.

[0080] The networking subsystem 214 includes one or more devices configured to couple to and communicate on (i.e., perform network operations over) a wired and / or wireless network, including: a control logic component 216, an interface circuit 218, and a set of antennas 220 (or antenna elements) in an adaptive array that can be selectively turned on and / or off by the control logic component 216 to produce a variety of optional antenna patterns or "beam patterns". (Although Figure 2 The antenna set 220 is included, but in some embodiments, the electronic device 200 includes one or more nodes, such as the node 208, for example, a pad, which can be coupled to the antenna set 220. Therefore, the electronic device 200 may or may not include the antenna set 220.) For example, the networking subsystem 214 may include a Bluetooth TM networking systems, cellular networking systems (such as 3G / 4G networks such as UMTS, LTE, etc.), universal serial bus (USB) networking systems, networking systems based on the standards described in IEEE 802.11 (e.g., networking system), an Ethernet networking system, and / or another network system.

[0081] The networking subsystem 214 includes a processor, a controller, a radio component / antenna, a socket / plug, and / or other devices for coupling to each supported networking system, communicating on each supported networking system, and processing data and events of each supported networking system. It should be noted that the mechanism for coupling to the network of each network system, communicating on the network of each network system, and processing data and events on the network of each network system is sometimes collectively referred to as a "network interface" for the network system. In addition, in some embodiments, the "network" or "connection" between electronic devices does not yet exist. Therefore, the electronic device 200 can use the mechanism in the networking subsystem 214 to perform simple wireless communication between electronic devices, such as transmitting announcement frames or beacon frames and / or scanning announcement frames transmitted by other electronic devices.

[0082] In electronic device 200, processing subsystem 210, memory subsystem 212, networking subsystem 214, display subsystem 226, measurement subsystem 230 and user interaction subsystem 232 are coupled together using bus 228, which facilitates data transmission between these components. Bus 228 may include electrical connectors, optical connectors, and / or optoelectronic connectors that can be used by these subsystems to transmit commands and data between each other. Although only one bus 228 is shown for clarity, different embodiments may include different numbers or configurations of electrical connectors, optical connectors, and / or optoelectronic connectors between subsystems.

[0083] In some embodiments, the electronic device 200 includes a display subsystem 226 for displaying information on a display, which may include a display driver and a display, such as a liquid crystal display, a multi-touch touch screen, etc. The display subsystem 226 can be controlled by the processing subsystem 210 to display information (e.g., information related to incoming, outgoing, or active communication sessions) to a user. In some embodiments, the display subsystem 226 can be configured to generate display information for a non-local display (e.g., a display associated with another device).

[0084] The electronic device 200 may also include a measurement subsystem 230 having one or more sensors that allow the electronic device 200 to perform one or more types of measurements. For example, the one or more sensors may include: one or more compasses, one or more accelerometers, one or more gyroscopes, one or more microphones or acoustic transducers, one or more environmental sensors (such as temperature sensors and / or altimeters), one or more light sensors (such as ambient light sensors), one or more touch sensors (such as touch screens), one or more biometric sensors (such as fingerprint sensors), etc. It should be noted that the one or more sensors may include physical sensors and / or virtual sensors (such as sensors implemented at least partially in software) in the electronic device 200. In some embodiments, at least some of the one or more sensors determine sensor data based on information received from a remote electronic device.

[0085] In addition, the electronic device 200 may include a user interaction subsystem 232. For example, the user interaction subsystem 232 may include a variety of user input devices, such as: buttons, keypads, dials, touch screens, audio input interfaces, visual / image capture input interfaces, input in the form of sensor data, etc. Alternatively or in addition, the user interaction subsystem 232 may include a variety of user output devices, such as: one or more speakers (which may provide a directional acoustic array), tactile transducers, etc.

[0086] The electronic device 200 may be any electronic device having at least one network interface (or may be included in any electronic device having at least one network interface). For example, the electronic device 200 may include: a cellular phone or smart phone, a tablet computer, a laptop computer, a notebook computer, a personal or desktop computer, a netbook computer, a computing device, a shared computing device (such as a printer), a media player device, an e-book device, a smart watch, a wearable computing device, a wearable device, a portable computing device, a consumer electronic device, an access point, a router, a switch, a communication device or communication device, a test device, an appliance, an entertainment device, a television, a display, a radio receiver, a set-top box, an audio device, a projector, a medical device (such as an automatic external defibrillator), a security device, an alarm, a monitoring device (e.g., a smoke detector or a carbon monoxide detector), an environmental controller, a thermostat, a light switch, an accessory, a keyboard, a mouse, a speaker, a printer, a home automation device, a vehicle, an electronic lock, and any other type of electronic computing device with wireless communication capabilities that may include communication via one or more wireless communication protocols.

[0087] Although specific components are used to describe the electronic device 200, in alternative embodiments, different components and / or subsystems may be present in the electronic device 200. For example, the electronic device 200 may include one or more additional processing subsystems, memory subsystems, networking subsystems, and / or display subsystems. In addition, one or more of these subsystems may not be present in the electronic device 200. Furthermore, in some embodiments, the electronic device 200 may include Figure 2 One or more additional subsystems not shown. Furthermore, although Figure 2 200, but in some embodiments, part or all of a given subsystem or component may be integrated into one or more of the other subsystems or components in the electronic device 200. For example, in some embodiments, the program module 222 is included in the operating system 224, and / or the control logic component 216 is included in the interface circuit 218.

[0088] In addition, the circuits and components in the electronic device 200 can be implemented using any combination of analog circuits and / or digital circuits, including: bipolar, PMOS and / or NMOS gates or transistors. In addition, the signals in these embodiments can include digital signals with approximately discrete values ​​and / or analog signals with continuous values. In addition, the components and circuits can be single-ended or differential, and the power supply can be unipolar or bipolar.

[0089] An integrated circuit (sometimes referred to as a "communication circuit") may implement some or all of the functionality of the networking subsystem 214. This integrated circuit may include hardware mechanisms and / or software mechanisms for transmitting wireless signals from the electronic device 200 and receiving signals from other electronic devices at the electronic device 200. Except for the mechanisms described herein, radio components are well known in the art and are not described in detail. In general, the networking subsystem 214 and / or the integrated circuit may include any number of radio components. Note that the radio components in the multi-radio embodiment function in a manner similar to the single radio embodiment described.

[0090] In some embodiments, the networking subsystem 214 and / or the integrated circuit includes a configuration mechanism (such as one or more hardware mechanisms and / or software mechanisms) that configures the radio component to transmit and / or receive on a given communication channel (such as a given carrier frequency). For example, in some embodiments, the configuration mechanism can be used to switch the radio component from monitoring and / or transmitting on a given communication channel to monitoring and / or transmitting on another communication channel. (Note that "monitoring" as used herein includes receiving signals from other electronic devices and possibly performing one or more processing operations on the received signals, for example, performing wireless ranging, etc.)

[0091] In some embodiments, the output of the process for designing an integrated circuit or a part of an integrated circuit including one or more of the circuits described herein may be a computer-readable medium, such as, for example, a programmable memory, a tape, or an optical or magnetic disk. The computer-readable medium may be encoded with a data structure or other information describing a circuit that can be physically instantiated as an integrated circuit or a part of an integrated circuit. Although various formats may be used for such encodings, these data structures are typically written in the following formats: Caltech Intermediate Format (CIF), Calma GDS II Stream Format (GDSII), or Electronic Design Interchange Format (EDIF). Technicians in the field of integrated circuit design are able to develop such data structures from the schematic diagrams and corresponding descriptions of the types described in detail above, and encode the data structures on a computer-readable medium. Technicians in the field of integrated circuit manufacturing can use such encoded data to manufacture integrated circuits including one or more of the circuits described herein.

[0092] The user interface technology can be used in a variety of network interfaces. In addition, although some of the operations in the aforementioned embodiments are implemented in hardware or software, generally speaking, the operations in the aforementioned embodiments can be implemented in a variety of configurations and architectures. Therefore, some or all of the operations in the aforementioned embodiments can be performed in hardware, software, or in hardware and software simultaneously. For example, at least some of the operations in the user interface technology can be implemented using program modules 222, operating systems 224 (such as drivers for interface circuits 218), or firmware in interface circuits 218. Alternatively or in addition, at least some of the operations in the user interface technology can be implemented in a physical layer such as hardware in interface circuits 218. In an exemplary embodiment, the user interface technology is implemented at least in part in the MAC layer in interface circuits 218.

[0093] In an exemplary embodiment, user interface technology is used to provide directional control. Specifically, when the position and orientation of the wireless communication device relative to the object is determined based on one or more measurements, one or more gesture inputs on or with the wireless communication device can be interpreted as commands for the object. As further described below, wireless ranging and sensor data (from gyroscopes, accelerometers, compasses, etc.) inputs can be used to select objects, pair with objects, and / or command objects.

[0094] In an exemplary embodiment, user interface technology is used to intuitively provide information about one or more objects in environmental control. Specifically, when the range and / or direction of an object is determined by a wireless communication device based on one or more measurements (such as wireless ranging and / or orientation), the wireless communication device can use the determined range and / or direction to intuitively provide output information about the object (such as in a user interface).

[0095] Control electronics

[0096] Figure 3 An illustration 300 is provided showing an example of controlling an object using gestures based at least in part on wireless ranging. Specifically, during the user interface technique, a user may use a wireless communication device 110 to control a physical device (e.g., physical device 112) by performing one or more gestures or movements (e.g., in two or three dimensions), or to generate control input for the physical device using interaction with a virtual representation of the object (e.g., virtual representation 114). The wireless communication device 110 is illustrated as generating gesture input relative to an object 310, which may be an actual device or a virtual representation. For example, a user may use their cell phone to control the volume or selected channel of a television, or turn on or off room lights, by performing gestures, such as by moving the wireless communication device 110 up and then down (or vice versa). Alternatively, the user may control Figure 1A predefined physical location (such as Figure 1 16) at a predefined physical location (i.e., there may not be a physical light switch at the predefined physical location). Note that the predefined physical location (and, more generally, the predefined physical location in three-dimensional space) may have been specified by a user using the wireless communication device 110 during a training process, such as by bringing the wireless communication device 110 into proximity with or touching the physical location and activating a virtual command icon in a user interface displayed on a touch-sensitive display on the wireless communication device 110, or by touching the wireless communication device 110 to the physical location and verbally speaking a command or instruction that is monitored by the wireless communication device 110 and interpreted using speech recognition technology.

[0097] During the user interface technique, the user may designate (or otherwise indicate) the object 310 using an “intention gesture” 312, such as by extending the wireless communication device 110 toward the object 310 or in the general direction of the object 310 (such as, for example, within 5° of the object 310). The user may then perform an “action gesture” 314, for example, by moving the wireless communication device 110 relative to the object 310. In one example, the wireless communication device 110 may move upward or downward relative to the object 310, for example, to signal an increase or decrease in a value, respectively. Other gestures may also be used, such as by providing lateral and / or rotational movement. In addition, gestures may be simple (e.g., involving one motion, such as vertical or horizontal motion) or complex (e.g., involving a combination of two or more motions, such as horizontal, vertical, lateral, and / or rotational motions). These operations may be performed while the user is pointing the wireless communication device 110 toward the object 310 or in the general direction of the object 310.

[0098] To detect the intended gesture 312, the wireless communication device 110 may measure its orientation relative to the position or location of the object 310. For example, the object 310 may be identified based on a universal or unique identifier transmitted by the object 310, and the relative position may be determined based on wireless ranging (e.g., using the wireless signal 122), a measurement (such as RSSI), and / or an orientation. In addition, to detect the motion gesture 314, the wireless communication device 110 may measure motion, acceleration, and / or distance 316, for example, using sensor input and / or wireless ranging. It is noted that the wireless ranging may be performed at or in one or more frequency bands, such as at or in the following frequency bands: a 2 GHz wireless band, a 5 GHz wireless band, an ISM band, a 60 GHz wireless band, an ultra-wideband band, etc.

[0099] Additionally, ambient sound may be used by the wireless communication device 110 to determine whether the object 310 is in the same environment 108 as the wireless communication device 110 ( Figure 1 ), such as in the same room (rather than an adjacent or neighboring room). For example, ambient sounds recorded by subject 310 that are shared with wireless communication device 110 via a wireless communication channel (such as WLAN) may be compared to ambient sounds measured by wireless communication device 110. If these are the same, then wireless communication device 110 and subject 310 may be located in the same room. Alternatively or in addition, wireless communication device 110 may output one or more acoustic frequency modulated signals or sounds (which may be at or in a frequency band outside of the frequency range audible to humans) that may be detected by subject 310, for example, if there are no intervening physical barriers.

[0100] More generally, one or more of the measurements performed by the wireless communication device 110 may be used to describe the environment 108 ( Figure 1 ) from one or more other devices, such as physical device 118. For example, object 310 facing a user of wireless communication device 110 may have the same orientation, a different height, and / or the same or different RSSI as another object in the environment. In combination with distance 316, the one or more measurements may be used to identify object 310 from all available devices. Additionally, as previously described, potential target devices may be eliminated based on ambient sounds. For example, ambient sounds may indicate that a device is in a pocket or bag and is therefore unlikely to be the target of intent gesture 312 for pairing or information sharing.

[0101] In some implementations, an altimeter can be used to exclude one or more potential target devices, such as potential target devices that are below a table or in a pocket / bag, when they are unlikely to be targeted by the intended gesture 312. As another example, one or more ultrasonic FM signals or 60 GHz transmissions can be used to exclude potential target devices that are within the primary radio range or wireless proximity of the wireless communication device 310, but outside of the associated physical space or physical proximity (e.g., the excluded potential target devices may be located in another room or stored in a box).

[0102] When two or more potential target devices cannot be separated in one or more of these ways, the user of the wireless communication device 110 may be required to select among a set of identified potential target devices. For example, a user interface with a list of potential target devices may be displayed on a display in the wireless communication device 110. The user may then select an intended target, such as intent gesture 312, from the interface and provide a desired action gesture 314 or other such control input.

[0103] While the foregoing discussion includes illustrative examples of intent gestures 312 and action gestures 314, in general, these gestures may include a variety of two-dimensional or three-dimensional motions and rotations, such as: linear motion in one or more directions, rotation about one or more axes, motion along an arc, virtual taps, number-based gestures (e.g., a figure 8 gesture), gestures in the shape of letters or numbers, combinations of linear motion and rotation, etc. More generally, action gestures 314 may have an intuitive correspondence with the type of variation or desired control operation. For example, Figure 4 As shown, the figure provides an illustration 400 showing an example of controlling an object using gestures, where an upward and / or downward motion gesture 410 can be used to increase (or decrease) a value or setting, such as a temperature setting of a heater or thermostat, or the position of an object (such as a window share or screen). Similarly, as Figure 5 As shown, the figure shows an example of controlling an object using gestures 500 , where the motion gesture 510 may involve left and / or right rotations corresponding to changing a value or setting, such as the volume of an entertainment device.

[0104] In one example, a user may point their cell phone at a television. Through wireless ranging, the cell phone may determine that the television is within range (i.e., close enough to be the intended target device). Accelerometer and / or gyroscope input may provide further indication of relative motion that may be interpreted as an intended gesture 312, thereby indicating that the wireless communication device 110 is being used to command (e.g., a motion gesture) the television, e.g., object 310. Additionally, one or more motion gestures (e.g., any of 314, 410, and / or 510) may be provided to indicate one or more commands for controlling the television. For example, a user may turn the cell phone to the right to increase the volume of the television or change the channel.

[0105] User interface techniques may also be used to intuitively pair the wireless communication device 110 with another electronic device so that the paired electronic devices can subsequently communicate with each other. For example, by performing an intention gesture 312 and an action gesture (314, 410, and / or 510), a user may be able to remotely indicate that it is desired to exchange or share data wirelessly with a physical device. When these operations are performed, the wireless communication device 110 and / or the physical device may display a message in the user interface, requesting confirmation and / or authorization for pairing. In addition, another physical device may agree to pairing for a specified time interval (e.g., 10 minutes or one hour), after which pairing may be disabled. In this way, the user of the wireless communication device 110 can maintain control of the pairing process, which can also help ensure that the pairing is safe, such as by confirming that the paired physical devices are in the same environment (such as the same room). By providing directional intention and action gestures, the wireless communication device 110 can avoid pairing with a device that is attempting to electronically deceive or imitate the object 310 intended to be paired with it. Similarly, directed intent and motion gestures can avoid accidentally pairing with a device that is close to the environment but at another location or height that is not object 310. This approach can also help reduce the complexity of sharing information and / or pairing. For example, a password may not be required during the pairing process. In some embodiments, after the intent gesture and the motion gesture, the wireless communication device and the physical device can exchange encryption keys.

[0106] In another example, the wireless communication device 110 may be pointed in the direction of an object with which it is intended to pair or to which a command is intended. A camera of the wireless communication device 110 may capture images of all objects in the field of view, and may present a selectable on-screen interface showing one or more candidate devices. The user may then select a device to pair with (and optionally also select, for example, a protocol).

[0107] In another example, the wireless communication device 110 may be pointed at the object 310 to request a resource or obtain information. For example, the wireless communication device 110 may be pointed at a taxi (e.g., object 310) to signal it to stop (e.g., using the intention gesture 312, with or without the action gestures 314, 410, and / or 510). In addition, the wireless communication device 110 may be pointed at a restaurant (or an object 310 associated with a restaurant) to obtain ratings or make a reservation (e.g., using the intention gesture 312, with or without the action gestures 314, 410, and / or 510). Similarly, the wireless communication device 110 may interact with a printer (e.g., set up or print) by pointing at the printer. For example, the intention gesture 312 and the action gestures 314, 410, and / or 510 may be provided at the wireless communication device 110 to send information to the printer (e.g., initiate a print job) or initiate control of one or more settings / configurations.

[0108] Wireless ranging and directionality may also be used to obtain information from one or more sensors. For example, the wireless communication device 110 may be pointed toward a sensor (e.g., a thermostat or tire pressure gauge) to obtain a reading. In some implementations, an intent gesture 312 with or without an action gesture 314, 410, and / or 510 may be used to initiate communication of sensor data. Similarly, one or more intent and / or action gestures may be provided at the wireless communication device 110 to retrieve information from a server, such as by interacting with an item or display in a store to check attributes, such as price or availability information.

[0109] In addition, identification of the object 310 by the wireless communication device 110, for example, may be facilitated by a (universal or unique) identifier selectively transmitted by the object 310. For example, the identifier may be a persistent unique identifier associated with a particular electronic device (e.g., object 310). The wireless communication device 110 may store the persistent unique identifier (e.g., locally or in the cloud) and may associate the identifier with a person, object, or location (e.g., in contact information). When the identifier is detected in the received wireless communication, the wireless communication device 110 may provide an indication that a known electronic device (e.g., object 310) is approaching. Alternatively, the identifier may be transient or temporary, such as an identifier assigned by an application executed on the wireless communication device 110 or the object 312. For example, a ride-sharing application may temporarily assign an identifier to the wireless communication device 110, which may then broadcast the identifier. In addition, the ride-sharing application may provide a transient identifier to a device corresponding to the assigned driver to facilitate pickup. Similarly, a (persistent or transient) identifier of a device corresponding to the driver may be provided to the wireless communication device 110 to verify the driver. A transient identifier may expire after a predetermined amount of time, such as one hour, or upon completion of a transaction. Any number of persistent and transient identifiers may be associated with a device or object, for example, to facilitate identification across a variety of scenarios.

[0110] In addition, the assigned identifier can change over time to protect the user's privacy and provide security. Specifically, the identifier can be Figure 1 The optional server 126 in Figure 1 The identifier may be assigned to a network 124 in the network (such as a wired or wireless network) and may be updated periodically (such as after a time interval, such as 5 minutes or 1 day). In addition or alternatively, the identifier may be assigned as a one-time identifier, such as for a specific transaction. In some embodiments, the identifier may be an anonymous identifier that may be persistently assigned to a device or object, such as object 310. The anonymous identifier can then only be resolved by other devices with which the anonymous identifier and identification information items have been shared.

[0111] To reduce power consumption, the wireless communication device 110 and / or the object 312 may transmit one or more identifiers only periodically or upon an event (such as detecting an identifier associated with a known contact). In addition or alternatively, the wireless communication device 110 and / or the object 312 may attempt to receive one or more identifiers only when one or more conditions are met (such as being near another (known or unknown) device, participating in a transaction involving another device, in a specific area (e.g., within a geo-fenced area), in response to user input, and / or based on an event (e.g., an appointment or meeting). For example, a Bluetooth Low Energy (BLE) receiver may periodically scan for announcements (or other messages) including an identifier from, for example, the object 312. The scanning rate may vary based on one or more factors, such as motion, time, and / or available power. In response to detecting a known identifier, the wireless communication device 110 may activate one or more other communication interfaces and may enable wireless ranging, such as so that the wireless communication device 110 can receive messages from the object 312 and / or determine a direction and / or distance 316 to the object 312.

[0112] In addition, as described above, user interface techniques may facilitate the use of virtual objects. These embodiments may include defining virtual objects and controlling them through wireless ranging / directional control inputs. In some embodiments, interaction with virtual objects (e.g., virtual representations 114) may be based on the orientation of the wireless communication device 110 and the received intention gestures without performing wireless ranging.

[0113] For example, a virtual object or virtual representation may be established in a specific physical location, such as a virtual light switch or virtual thermostat next to a door. A virtual object may be physically represented by a visual item (e.g., a sticker), or may be purely virtual, corresponding only to one or more coordinates. A virtual object may also be defined by an associated physical device. For example, a projector may define a virtual control, such as on a wall, and may announce the location of the virtual control, thereby allowing other electronic devices to manipulate the virtual control to implement actual input to the projector.

[0114] As previously described, in embodiments with virtual objects, a device (e.g., wireless communication device 110) may define a virtual device at a specific location, such as a virtual control corresponding to a specific location on a wall. Note that a virtual device may be defined as part of a larger system. For example, a facility may define a virtual thermostat adjacent to each entry door (e.g., at a light switch). Alternatively, a virtual device may be defined by a specific device, e.g., a projector may define a control on a wall or other surface, and the location of the control may be wirelessly announced, allowing other electronic devices to manipulate the virtual control as if it were real.

[0115] In this way, the wireless communication device 110 (e.g., a cellular phone or a wearable device) may be able to be used as a control device, such as a dynamic remote control. In addition, the wireless communication device 110 may be able to provide control inputs for a variety of objects (e.g., object 310) through gestures without being configured with a specific input mechanism. This may allow the display of the wireless communication device 110 to act as a proxy touch screen for a larger monitor, and sensors / inputs such as a compass, gyroscope, accelerometer, altimeter, touch input and / or microphone may be used to control input and / or output associated with the monitor. In some embodiments, the wireless communication device 110 may be used as a second screen corresponding to the main display (note that it may or may not mirror the main display). Thus, gesture-based (including touch-based) controls may be provided to control the main display (e.g., pinch, zoom, scroll, swipe, etc.).

[0116] In addition, for example, gesture-based control inputs to the wireless communication device 110 may be used to perform functions, such as content transfer between electronic devices. For example, one or more gesture-based control inputs to the wireless communication device 110 may be used to send (e.g., "cast") content to a second electronic device, pull content from a second electronic device, take control of a second electronic device (e.g., take ownership of a computer and have it copy your content / settings), and the like. Thus, one or more gesture-based inputs (e.g., flicking, pushing, rubbing, swiping, or tapping) may be used to indicate a transfer from the wireless communication device 110 to another device such as a larger display or computer (or from another device to the wireless communication device 110). Similarly, using the wireless communication device 110 as a data carrier, gesture-based inputs to the wireless communication device 110 may be used to move something from one device to another.

[0117] In some specific implementations, virtual objects may encompass constructs, such as virtual queues (or lines), for example, for services. For example, a virtual queue may be defined in association with an entity or resource (e.g., a ticket office or store), and nearby electronic devices may be registered in the virtual queue by transmitting an associated identifier (e.g., a unique transient identifier) ​​to a host device that maintains the virtual queue. The virtual queue position of a device (e.g., a wireless communication device 110) may be maintained or maintained until the queue is processed when the device remains near the physical location of the virtual queue. In addition, the position in the virtual queue may also be presented to the registered device and may be updated. The host device may notify the registered device when its (e.g., at the ticket office) turn arrives or approaches.

[0118] In addition to cost savings, using virtual representations or virtual objects for corresponding physical devices can also allow the environment to be more aesthetically pleasing by replacing less attractive physical devices (such as light switches) with attractive placeholders (e.g., ceramic objects) or by removing physical devices entirely. This approach also allows the virtual representation to be moved arbitrarily by simply redefining the location associated with the virtual representation. Then, when the user performs intention gestures and action gestures while pointing their electronic device, such as wireless communication device 110, toward the new location, the associated operations can be performed.

[0119] In some embodiments, gestures for controlling associated physical devices can be implicit. In such embodiments, proximity determined by wireless ranging can be used to engage controllable devices without associated intent and action gestures. For example, a light can be turned on (and then off) when a user walks past a predefined location associated with a virtual representation or virtual object. Similarly, a door lock can be unlocked when a user device approaches a door, a traffic light can turn green when the user device reaches an intersection (if conditions permit), and so on. Thus, the disclosed technology can be used for proximity-based control.

[0120] In addition, the disclosed technology can also be used to provide collaborative situational awareness, such as when an electronic device ahead in the direction of travel experiences an event, such as experiencing a sudden deceleration. The electronic device can notify other electronic devices of a potential dangerous situation, such as by broadcasting a message.

[0121] In some embodiments, the disclosed technology is provided by environment 108 ( Figure 1 ) in one or more optional tags such as optional tag 128 ( Figure 1 ) is convenient. For example, the tag can be a low-cost system-on-chip that harvests power from RF energy and / or has an associated power supply. The tag 128 can output one or more signals, for example, in response to a received command. For example, the tag 128 can output an RF signal and / or an acoustic signal (such as a "frequency modulated signal").

[0122] In some embodiments, a tag 128 ( Figure 1) may collect power in the air (e.g., it may receive power in the form of a 24 GHz signal) and communicate via a short signal or chirp signal (e.g., by transmitting a 60 GHz signal). The communication may have a range of up to, for example, 100 feet, and may be directional. In some implementations, the 24 GHz power signal may have an identifier encoded in the signal. In such implementations, if the tag does not recognize the identifier, it may not respond. The time period for the tag to respond to the start pulse (e.g., once it has indicated that it is powered and operating) may be known. Note that the wireless communication device may measure the phase and angle of arrival to determine the direction, and the time of arrival or time of flight may be used to determine the distance between the wireless communication device and the tag. Thus, the tag 128 may be used to generate or supplement wireless ranging calculations.

[0123] Figure 6 To illustrate a method for utilizing a wireless communication device such as Figure 1 Flowchart of an example of a method 600 for controlling an object using a wireless communication device 110 in FIG. During operation, the wireless communication device identifies a received intention gesture (operation 610) indicating an intention to control an object positioned proximate to the wireless communication device. The wireless communication device then determines an object associated with the intention gesture (operation 612). For example, the wireless communication device uses wireless ranging to determine an object within range that is also located in a direction and / or altitude corresponding to the position and / or orientation of the wireless communication device, for example, based at least in part on the intention gesture. In addition, the wireless communication device accesses sensor data associated with one or more sensors of the wireless communication device (operation 614) and interprets the sensor data to determine a command value (operation 616). For example, the wireless communication device compares the sensor data with one or more motion gestures (e.g., based on a motion gesture profile stored on the wireless communication device) to identify one or more intended commands. The wireless communication device then generates and transmits the command value, for example, using one or more messages, to control the object (operation 618). In some specific implementations, the one or more messages may be transmitted to the object based on an identifier advertised by the object or otherwise associated with the object.

[0124] In some embodiments of method 600, there may be additional, different, and / or fewer operations. Furthermore, the order of the operations may be changed, and / or two or more operations may be combined into a single operation or performed in parallel.

[0125] User interface technology implementation in Figure 7 As further illustrated in Figure 7A diagram 700 is shown illustrating communication between a wireless communication device 110 and a physical (or electronic) device 112 (which is used as an illustrative example). A user may perform an intended gesture 708, for example, by pointing or orienting the wireless communication device 110 toward the physical device 112. In some implementations, the intended gesture 708 may include an orientation component and a motion component, such as a motion to propel the wireless communication device 110 toward the physical device 112. A processor 714 in the wireless communication device 110 may identify the intended gesture 708, for example, based on the orientation 710 and any other associated motion measured by one or more sensors 712 (e.g., a gyroscope, accelerometer, and / or compass). The intended gesture 708 may be interpreted as a user's intent to control the physical device 112.

[0126] The interface circuit 718 in the wireless communication device 110 receives a packet 720 (or frame) transmitted by the interface circuit 716 in the physical device 112. The packet 720 includes an identifier 722 associated with the physical device 112. In addition, the packet 720 may include information specifying the transmission time of the packet 720. In conjunction with the determined arrival time of the packet 720 (based on the propagation speed of the wireless signal transmitting the packet 720), the interface circuit 718 (or the processor 714) may determine a distance 724 from the physical device 112. In addition or alternatively, the wireless communication device 110 may communicate one or more other packets with the physical device 112 to perform wireless ranging operations. Based on the identifier 722 and / or the distance 724, the processor 714 may identify the physical device 112.

[0127] In addition, the motion gesture may be performed at the wireless communication device 110 corresponding to a command or command value. One or more of the sensors 712 measure sensor data 726 during the motion gesture and may store the measured sensor data in the memory 728. The processor 714 may access the sensor data 726 (e.g., by receiving from the sensor 712 and / or retrieving from the memory 728) and interpret the sensor data 726 to determine a command value 730 corresponding to the motion gesture. In addition, the interface circuit 718 may transmit a packet 732 including the command value 730 to the interface circuit 716. Upon receiving the packet 732, the physical device 112 may implement the command value 730.

[0128] Augmented Reality

[0129] Figure 8 A diagram 800 is shown illustrating an example of locating an object in an environment using wireless ranging. Specifically, Figure 1 A physical device 112 in the environment 108 in proximity to the wireless communication device 110 may transmit a packet (or frame) having an identifier associated with the physical device 112. The wireless communication device 110 may then receive the transmission including the identifier.

[0130] For example, the identifier may be a universal identifier or a unique identifier for the physical device 112 (and therefore, a universal identifier or a unique identifier for the user of the physical device 112). Alternatively, the identifier may be associated with a resource class corresponding to the physical device 112, such as: transportation resources, shared computing resources (e.g., a printer), medical resources, display resources, security resources, accessibility resources, or safety resources. Note that the identifier may have been received in conjunction with a search result or a desired financial transaction (such as when the user wants to purchase a particular item available in proximity to or associated with the physical device 112). Specifically, if a user of the wireless communication device 110 performs a search for a resource class (e.g., using a search engine) and a transmission is received, the wireless communication device 110 may display location information associated with the physical device 112 at the time the identifier was received, as further described below. Alternatively or in addition, the identifier may correspond to a contact of the user of the wireless communication device 110, such as a friend or colleague of the user (i.e., someone the user already knows), or a person the user is connected to in a Figure 1 The user may or may not know the person to be met in the environment 108 in the . For example, to facilitate the meeting, either or both parties may provide an identifier in advance, so that the corresponding device can be monitored for it.

[0131] In some embodiments, the identifier may be associated with persistent contact information of another person stored on the wireless communication device 110 (or accessible in a remotely stored data structure) (e.g., associated with the physical device 112). Then, when the identifier is detected by the wireless communication device 110, an indicator may be provided to signal the presence and identity of the person corresponding to the identifier. Alternatively, the identifier may be transient or temporary, such as an identifier assigned by an application executed on the wireless communication device 110 or the physical device 112. For example, a ride-sharing application may temporarily assign an identifier to the wireless communication device 110, which may then broadcast it. In addition, the ride-sharing application may provide a transient identifier to the device (e.g., the physical device 112) corresponding to the assigned driver to facilitate pickup. Similarly, a (persistent or transient) identifier of the device corresponding to the driver may be provided to the wireless communication device 110 to verify the driver. The transient identifier may expire after a predetermined amount of time (such as one hour) or when the transaction is completed. Any number of persistent and transient identifiers may be associated with a device or object, for example, to facilitate identification across a variety of scenarios.

[0132] In addition, the identifier may change over time to protect the user's privacy and provide security. Specifically, the identifier may be generated by the cloud-based server 126 via Figure 1The identifier may be assigned by a network 124 (such as the Internet) in the network and may be updated periodically (such as after a time interval, such as 5 minutes, has passed) or may be a one-time identifier, such as associated with a transaction. In some embodiments, the identifier is an anonymous identifier that may be persistently assigned to, for example, a physical device 112. However, the anonymous identifier may be known to one or more contacts, which may resolve the anonymous identifier, for example based on locally stored associations. Note that an electronic device may be configured with several identifiers that may be transmitted periodically. This last capability may be used in one or more of the aforementioned embodiments.

[0133] In response to receiving the identifier, the wireless communication device 110 may compare the identifier to one or more stored target identifiers in a local (or remote) memory to determine a match. Thus, using wireless ranging, the wireless communication device 110 may determine a range (or distance) 810 of the physical device 112 relative to the wireless communication device 110 and / or a direction 812 to the physical device 112. Alternatively or in addition, the wireless communication device 110 may determine the range 810 and / or direction 812 based on a measurement (such as RSSI) and / or an orientation of the wireless communication device 110. It is noted that the wireless ranging may be performed at or in one or more frequency bands, such as at or in the following frequency bands: a 2 GHz wireless band, a 5 GHz wireless band, an ISM band, a 60 GHz wireless band, an ultra-wideband band, etc.

[0134] To reduce power consumption, the wireless communication device 110 and / or the physical device 112 may communicate only when they are in close proximity (such as when both are in Figure 1 The wireless communication device 110 may transmit a unique or universal identifier or attempt to receive a unique or universal identifier only when the wireless communication device 110 is in the environment 108 in the wireless communication device 110 (and more generally, when they are in a geo-fenced area) and / or based on an appointment or meeting time between users of the wireless communication device 110 and / or the physical device 112. For example, a Bluetooth low energy receiver may be used to receive a message including an identifier from, for example, the physical device 112. Based on receiving the message, the wireless communication device 110 may wake up one or more receivers and initiate wireless ranging so that the wireless communication device 110 can determine the range 810 and / or direction 812 to the physical device 112. In some specific implementations, the wireless communication device 110 may also receive a message with a unique identifier from the physical device 112.

[0135] Then, refer to the following Figure 9-Figure 15 As further described, the wireless communication device 110 may present output information indicating a range and / or direction to an object, such as the physical device 112 .

[0136] In some embodiments, the ambient sound may be used by the wireless communication device 110 to determine whether the physical device 112 is in the same environment 108 as the wireless communication device 110. Figure 1 ), such as in the same room (rather than an adjacent or neighboring room). For example, ambient sound recorded by the physical device 112 that is shared with the wireless communication device 110 via a wireless communication channel (such as a WLAN) may be compared to ambient sound measured by the wireless communication device 110. If the ambient sound measurement compares favorably, the wireless communication device 110 and the physical device 112 may be determined to be in the same room. Alternatively or in addition, the wireless communication device 110 may output one or more acoustic frequency modulated signals or sounds (which may be at or in a frequency band outside the range of human hearing), which may be detected, monitored, and / or recorded by the physical device 112.

[0137] More generally, one or more measurements performed by the wireless communication device 110 may be used to describe the environment 108 ( Figure 1 ), such as distinguishing the physical device 112 from one or more other devices. For example, target devices facing a user of the wireless communication device 110 may have the same orientation, different heights, and / or the same or different RSSIs. In conjunction with the identifier and / or range 310, the one or more measurements may be used to identify the physical device 112. In addition, as previously described, potential target devices may be eliminated based on ambient sounds, which may indicate that they are in a person's pocket and are therefore unlikely to be of interest for pairing or information sharing.

[0138] Altimeter or angle of arrival information may also be used to exclude potential target devices outside of the area of ​​possible interest, such as below a table or in a pocket / bag, when they are unlikely candidates to be located. As another example, ultrasonic FM signals or 60 GHz transmissions may be used to exclude potential target devices that are within the primary radio range or wireless proximity, but outside of the associated physical space or physical proximity (e.g., an excluded potential target device may be located in another room).

[0139] However, if multiple potential target devices cannot be separated in one or more of these ways, the user of the wireless communication device 110 may be required to select among a set of identified potential target devices. For example, a user interface with a list of potential target devices may be presented on a display of the wireless communication device 110, and a device may be selected by user input.

[0140] Although physical device 112 is used as an illustrative example in the foregoing discussion, in other embodiments, objects ( Figure 1 ) can be a virtual representation, such as Figure 1 For example, the virtual representation may be a virtual representation 114 of a physical device 118 in the Figure 1 A predefined physical location in a room or environment 108 (such as Figure 1 Note that the predefined physical location (and, more generally, the predefined physical location in three-dimensional space) may have been specified by a user, for example, via the wireless communication device 110 during a training process, such as by bringing the wireless communication device 110 close to the physical location or touching the physical location and activating a virtual command icon in a user interface displayed on a touch-sensitive display on the wireless communication device 110, or by touching the wireless communication device 110 to the physical location and verbally speaking a command or instruction that is monitored by the wireless communication device 110 and interpreted using speech recognition technology.

[0141] We now describe examples of displayed information based on the determined position of an object in the environment, such as the physical device 112 . Fig. 9 Presentation shows an example of a user interface 900 on a display 908 that displays a map 910 of a proximity area in an environment (and more generally, a graphical representation of the environment). When a user holds the wireless communication device 110 in the palm of their hand, the map 910 may be displayed. The map 910 may include an indicator 912 representing the location of an object such as a physical device 112. In some specific implementations, the location of the indicator 912 alone may convey the determined range and / or direction to the corresponding object. In some other specific implementations, additional information representing the determined range and / or distance may also be presented in conjunction with the map 910.

[0142] Note that map 910 may provide intuitive situational awareness of objects (such as physical device 112) and their relative locations in the environment surrounding wireless communication device 110. For example, user interface techniques may allow a user to identify the location of resources in the environment, such as fire extinguishers, first aid kits, automated external defibrillators, exits, and / or people with medical training. Alternatively or in addition, user interface techniques may provide dynamic location information about approaching objects, such as when an electronic device ahead on a path of travel experiences an event (e.g., a sudden deceleration). The event may be notified to, for example, one or more electronic devices approaching the electronic device on the path of travel.

[0143] Fig.10A diagram is presented showing an example of a user interface 1000. The user interface 1000 is presented on the display 908 of the wireless communication device 110 and may display an image 1010 of a proximity area captured by an image sensor (such as a CMOS or CCD imaging sensor, and more generally, a camera) of the wireless communication device 110. The wireless communication device 110 may also display annotations or indicators 1012 (which may be associated with the image 1010) superimposed on the image 1010. Fig. 9 912 in the wireless communication device 110). Fig. 9 Indicator 912 in ) may be displayed regardless of whether the corresponding object is within the visual range of the wireless communication device 110 or otherwise visible to the wireless communication device 110. For example, indicator 1012 may be presented to help a user of the wireless communication device 110 locate an object that is difficult to see or cannot yet be seen.

[0144] Thus, user interface techniques may include or may use augmented reality. Specifically, location information (such as range and / or direction) determined using wireless ranging may be superimposed on an image of the surrounding environment. This augmented reality may provide added intelligence to help the user of the wireless communication device 110 identify nearby things, locations, resources, and / or people. For example, this method may be used to assist the user in locating people they want to meet (people-based), locating resources such as first aid or wireless coverage (resource-based), and / or finding results of location-based searches such as buildings or streets (search-based). The display on the wireless communication device 110 may include a camera input that is enhanced to display items of interest.

[0145] Thus, if a user is in a crowded environment and they hold up the wireless communication device 110, an image of the person they are looking for can be displayed over the image of the environment when the wireless communication device 110 is oriented or pointed toward that person (or their associated electronic device, such as the physical device 112). Fig.16 As further described, when the user rotates or turns (e.g., based on the measured orientation of the wireless communication device 110), their associated smartwatch or other such wearable device may provide tactile feedback, such as a "tap" (or another type of feedback, such as an audio output) when they are facing the correct direction - e.g., the direction of the person, object, or resource they are looking for. Thus, the user interface technology may facilitate social interactions and encounters, even in crowded environments and even when the user has not met the person they are looking for before.

[0146] In some embodiments, the wireless communication device 110 automatically switches or transitions back and forth between different user interface modes. Fig.11 It is shown in Fig.11 A diagram is presented illustrating examples of different modes of a user interface based on the orientation of the wireless communication device 110. Specifically, when a user of the wireless communication device 110 changes the orientation of the device, the gyroscope or accelerometer may measure changes in the orientation value. Then, based on the measured orientation value, the wireless communication device 110 may selectively change the orientation of the user interface when displaying the map 910 ( Fig. 9 ) and display image 1010( Fig.10 ). Thus, when the wireless communication device 110 is oriented horizontally, such as in the palm of a user's hand, the displayed map 910 ( Fig. 9 ) can be displayed. When the wireless communication device 110 is oriented vertically, such as facing the direction of the physical device 112, the display image 1010 ( Fig.10 ) can be displayed. A transition between the map view 1110 and camera view 1112 interface modes can be initiated when one or more sensor inputs indicate an orientation change that exceeds a threshold amount. Alternatively or in addition, the map view 1110 can be displayed when the physical device 112 is out of the user's visual range, and the "camera view" 1112 can be displayed when the physical device 112 is in the visual range. In this way, the user can intuitively switch or transition back and forth between the two user interface modes by changing the orientation of the wireless communication device 110.

[0147] Alternatively, in some embodiments, a user of the wireless communication device 110 may manually select or specify a user interface mode. Fig.12 It is shown in Fig.12 An illustration of examples of different modes of the example user interface 1200 is presented. For example, by providing touch input to a touch-sensitive display of the wireless communication device 110 within a tap area of ​​the virtual icon 708, the user may select a map view having a map 1210 showing, for example, a 360° view around the wireless communication device 110. The map 1210 may show one or more objects (such as the physical device 112) at different ranges and directions (such as the range 1212 and the direction 1214), which are represented by indicators (such as the indicator 1216), for example, any / all of the following: text, thumbnails, symbols, emoticons, or graphical icons. In addition, when the wireless communication device 110 is oriented vertically and facing in the direction 1214 toward the physical device 112, the user may select an image view in which an image such as the image 1010 ( Fig.10 ) is displayed, and the physical device 112 is represented by an indicator 1012 (e.g., any / all of the following: text, thumbnail, symbol, emoticon, or graphical icon).

[0148] In some embodiments, user interface techniques may include additional variations on augmented reality. Fig.13 , which presents an illustration of an example of augmented reality in an example user interface 1310, a forward-facing camera on a wireless communication device 110 may capture an image (or video) of the environment in front of the wireless communication device 110. When the wireless communication device 110 is facing in the direction of an object, an image (such as image 1312) may be displayed on the wireless communication device 110 along with one or more indicators 1314 representing the object (such as physical device 112). Thus, the wireless communication device 110 presents the environment in front of the user in a manner enhanced with additional information. Note that the indicators 1314 may be positioned in the image 1312 so that they represent the respective locations of their corresponding objects in the environment 108. In addition, the size of the indicators 1314 may be set to provide a perspective view with intuitive information about the relative range to the corresponding object. Thus, objects that are farther away from the wireless communication device 110 (and therefore from the user) may be represented by smaller indicators than objects that are closer to the wireless communication device 110.

[0149] Fig.14 A diagram illustrating an example of extended situational awareness about objects in an environment is presented, including depicting objects that may be located out of sight. Specifically, when an object, such as a physical device 112, is located on the other side of an opaque barrier, such as a wall, one or more images 1410 may be displayed in a user interface 1400 of a wireless communication device 110 to allow the user to see the physical device 112 as if the opaque barrier was not there (or as if the user could see through the opaque barrier). For example, if a user is looking for a resource (such as a printer) in another room, an image 1410 showing the resource may be displayed in the user interface 1400 when the user points the wireless communication device 110 in the direction of the resource. In some implementations, the image 1410 may include additional information about the resource (e.g., the physical device 112). For example, the additional information may allow the user to physically or virtually access the resource.

[0150] Thus, the user interface may include instructions on how to reach the resource (and optionally also a map). The display may present a map indicating where the requested object is located, for example, based on receiving an identifier from the object and determining a range and / or direction. Alternatively or in addition, the location information may be obtained by searching. For example, when the requested object is out of visual range, the method may be used to guide the user to the object, such as displaying a route through an airport to ground transportation, customs, etc. Similarly, the method may be used to help the user move through a shopping mall or store by enhancing the map or display with directions to, for example, the desired object, resource, or location. In addition, the augmented reality method may annotate areas with Wi-Fi coverage, such as coverage provided by a specific operator or service provider. Note that the augmented information may be applied on an individual basis (e.g., meeting another person for whom the user has identification information such as a phone number); on a search basis (e.g., the user searches for and obtains the coordinates of a place / thing); on a resource basis (e.g., first aid, security, toilets may announce their locations, taxis, or restaurant locations), or a combination thereof.

[0151] Alternatively or in addition, the user may instruct the wireless communication device 110 to pair or connect with the physical device 112 by tapping an icon associated with the physical device 112 in the user interface 1400 displayed on the wireless communication device 110. This intuitive pairing may allow the devices to subsequently communicate with each other. Thus, a resource (such as a computer in a crowded workspace) may be selected through touch input to the user interface 1400.

[0152] When these operations are performed, the wireless communication device 110 and / or the physical device 112 may display a message in the user interface 1400, which allows either or both of the users of the wireless communication device 110 and / or the physical device 112 to confirm the pairing. In addition, the pairing with the physical device 112 may be continued for a specified time interval, such as 10 minutes or one hour, after which the pairing may be interrupted. In this way, the user and / or resource can maintain control of the pairing process, which can also help ensure that the pairing is safe by confirming that the pairing is with a physical device in the same environment (e.g., the same room). In this way, the user can avoid accidentally pairing with a nearby device that is deceiving or otherwise simulating another device. This method can also help reduce the complexity of sharing information and / or pairing. For example, a password may not be required during the pairing process. In some embodiments, after the above-mentioned virtual pairing operation, devices such as the wireless communication device 110 and the physical device 112 can exchange encryption keys.

[0153] In another example, the wireless communication device 110 may point in the direction of an object to be paired with. The camera of the wireless communication device 110 may capture images of all objects in that direction and may provide a selectable on-screen indicator showing pairing candidates. For example, the object may be determined based at least in part on an associated wirelessly transmitted identifier and one or more ranging operations. The user may then select an item (and protocol) to be paired with using the on-screen indicator.

[0154] like Fig.15 As shown, user interface techniques may also be used to provide reminders and / or track objects in an environment. For example, user interface techniques may allow a user to track the location of an object (such as physical device 112) in an environment, or to obtain reminders about groups of related objects in an environment. In this way, a parent may keep track of a child or an individual may ensure that one or more objects are not forgotten.

[0155] In some embodiments, a user bundles a group of tagged related items (such as a tent and associated tent poles). If not all items are in the group, for example, if the user forgets an item, feedback such as an alert is provided to the user. In addition, user interface techniques can be used to track items tagged with unique identifiers. For example, a camera can be configured to track a child on stage during a performance. Similarly, user interface techniques can be used to remind users not to leave tracked / tagged items behind when leaving an area. For example, if an umbrella is outside a threshold distance, a reminder can be issued. In addition, these techniques can be used to track the location / movement of items in an area, track users for gaming purposes (e.g., virtual paintball), etc. For example, if a user moves more than a predefined distance relative to a tagged item, he or she can receive a reminder. More generally, these techniques can involve using augmented reality to remind users of things they see or things they need to do, such as names, errands, etc.

[0156] As previously described, the reminder function can be facilitated by one or more optional tags such as optional tag 1510 in environment 108, which is included in or near an object of interest (such as physical device 118). Specifically, optional tag 1510 can collect power in the air (for example, it can receive a 24GHz signal) and can communicate through a chirp signal or a short transmission (for example, 60GHz). The communication can have a range of up to, for example, 100 feet and can be directional. The 24GHz power signal can also include an identifier encoded on the signal. If the optional tag 1510 does not recognize the signal, it can not respond. The time period for the optional tag 1510 to respond to the start pulse (for example, once it has indicated that it is powered and operating) can be known. It should be noted that the wireless communication device 110 receiving the signal from the tag 1510 can measure the phase and angle of arrival to determine the measurement of the direction, and the arrival time or flight time can be used based on the known response time to determine the range between the wireless communication device 110 and the tag 1510.

[0157] Various feedback can be provided to users or other individuals in an environment using user interface techniques. Fig.16 It is shown in Fig.16 A diagram is presented showing examples of different types of feedback associated with a user interface. Specifically, in addition to a visual display of determined location information, the wireless communication device 110 can provide other types of feedback to the user. For example, the wireless communication device 110 can provide tactile feedback directly through the wireless communication device 110 or indirectly through another electronic device (such as a user's smart watch). Specifically, the wireless communication device 110 can instruct the smart watch 1610 to provide tactile feedback via the transducer (trans.) 1612. In this way, the user can receive feedback when facing the direction of a tracked object, such as the physical device 112.

[0158] In addition, the wireless communication device 110 may transmit a message, such as a text or audio message, to the physical device 112. For example, the message may include an audio message that is transmitted as a beamformed audio message 1614 that is substantially only perceptible at a location corresponding to the physical device 112. In this way, a user of the wireless communication device 110 can send a private message to another individual within range (e.g., a person that the user can see), even when the user does not have conventional contact information (such as a phone number or email address) for an electronic device associated with that person. For example, a message may be directed to a target device based on an identifier (e.g., an anonymous identifier) ​​received from the target device, such as in a notification beacon. Thus, such directional communications may be anonymous. In some embodiments, the beamformed audio 1614 message may be transmitted via infrastructure in a building or structure.

[0159] Alternatively or in addition, the wireless communication device 110 may transmit a beamformed audio message with useful information to the user of the wireless communication device 110, such as the names of people in the environment, errands to be run, upcoming appointments, etc. Thus, augmented reality may be implemented using directional communications, such as by alerting the user to things in the environment through visual queues and beamformed audio messages that can only be perceived by the user of the wireless communication device 110.

[0160] Note that tactile feedback and / or messaging can enhance the search functionality by confirming directionality, e.g., the wireless communication device 110 can emit a beep and / or buzz when the wireless communication device 110 is pointed in the correct direction. Alternatively or in addition, the tactile feedback can come from a second device, such as an associated smartwatch. The second device can output tactile feedback to notify the user of conditions or events, such as when the wireless communication device 110 is pointed in the direction of the sought resource; a vehicle that is indicated to be stopped is approaching; to highlight a sign / address that is difficult to see; to indicate that the bus / train is what the user needs and is going to where the user wants to go, etc.

[0161] In some embodiments, user interface technology is used to provide additional features, such as: location-based anonymous chat rooms; an electronic assistant that can respond to searches or read emails so that only the device user can perceive (e.g., hear) the responses; to determine whether the user has friends in the room; etc.

[0162] Fig.17 To illustrate a method for utilizing a wireless communication device such as Figure 1 1700 for locating an object using a wireless communication device 110 in FIG. 1701. During operation, the wireless communication device receives a transmission having an identifier associated with an object located proximate to the wireless communication device (operation 1710). In some embodiments, the wireless communication device optionally compares the received identifier with one or more stored target identifiers to determine if a match exists (operation 1712).

[0163] The wireless communication device determines a range and / or direction from the wireless communication device to the object (operation 1714). The range, direction, or both are determined at least in part using one or more wireless ranging techniques. The wireless communication device then presents output information indicating the range and / or direction of the object relative to the wireless communication device (operation 1716). For example, the wireless communication device may: display a map with a first indicator representing the object or display an image with a second indicator representing the object. In some implementations, the wireless communication device may, for example, transmit a message to the object using an identifier included in a transmission from the object.

[0164] In some embodiments of method 1700, there may be additional or fewer operations. For example, in some embodiments, the wireless communication device does not receive a transmission with an identifier (operation 1710). In addition, the order of the operations may be changed, and / or two or more operations may be combined into a single operation or performed in parallel.

[0165] User interface technology implementation in Fig.18 As further illustrated in Fig.18 A diagram illustrating communication between a wireless communication device 110 and a physical device 112 is presented (which is used as an illustrative example). An interface circuit (IC) 1812 in the wireless communication device 110 may receive a message 1810 from an interface circuit 1808 in the physical device 112. The message may include an identifier 1814 associated with the physical device 112 and / or a user of the physical device 112. The identifier 1814 is provided to a processor 1818. In some embodiments, the processor 1818 optionally accesses one or more stored target identifiers 1820 in a memory 1822 and optionally compares the identifier 1814 to the one or more stored target identifiers 1820 to determine if a match 1824 exists.

[0166] The interface circuit 1812 may then utilize one or more wireless ranging and / or angle of arrival techniques to determine a range 1826 and / or a direction 1828 relative to the physical device 112. The range 1826 and direction 1828 are provided to the processor 1818.

[0167] The processor 1818 may then present output information indicating the range 1826 and / or the direction 1828. For example, the processor 1818 may display information 1830 on a display 1832 or provide display instructions to the display 1832, such as a map with a first indicator representing the physical device 112 or an image with a second indicator representing the physical device 112. Alternatively or in addition, the processor 1818 may optionally provide another type of feedback, such as by sending tactile feedback information 1834 to the transducer 1836 to generate tactile feedback 1838. In some embodiments, the processor 1818 provides one or more instructions 1840 to the interface circuit 1812, which transmits a message 1842 (e.g., a text message) to the physical device 112 based on the identifier 1814. The message 1842 may include one or more information items, including either or both of the range 1826 and / or the direction 1828.

[0168] Representative Implementation Plans

[0169] In some embodiments, a method for controlling an object implemented by a wireless communication device includes: (i) identifying, by the wireless communication device, an intention gesture indicating an intention to control the object, wherein the object is positioned proximate to the wireless communication device; (ii) the wireless communication device determining the object associated with the intention gesture based at least in part on a wireless ranging operation; (iii) accessing sensor data generated by one or more sensors associated with the wireless communication device; (iv) interpreting the sensor data to determine a command value; and (v) transmitting the command value to control the object.

[0170] In some embodiments, the object comprises a physical object. In some embodiments, the physical object comprises a computing device, a display, a printer, a communication device, an audio device, an appliance, a wearable device, a home automation device, an environmental controller, or an accessory. In some embodiments, the object comprises a virtual object. In some embodiments, the virtual object comprises a proxy of a physical object at different locations. In some embodiments, the wireless ranging operation comprises determining the distance between the wireless communication device and the object using wireless transmission characteristics. In some embodiments, the accessed sensor data comprises an orientation value corresponding to the wireless communication device, the orientation value being determined using at least one of a compass, an accelerometer, or a gyroscope. In some embodiments, the command value comprises a command for pairing the wireless communication device with the object. In some embodiments, the command value comprises a command for changing the operational settings of the object. In some embodiments, the command value comprises a command for performing a function. In some embodiments, the method further comprises determining the proximity of the object using an ultrasonic chirp signal. In some embodiments, the method further comprises receiving a response comprising a sensor value from the object in response to the command value. In some embodiments, the command value comprises a request for accessing a resource.

[0171] In some embodiments, a wireless communication device includes: (i) an interface circuit configured to communicate wirelessly with an object using at least one wireless communication protocol; (ii) one or more sensors configured to measure sensor data; (iii) a processor communicatively coupled to the interface circuit and the one or more sensors, the processor configured to execute a program module; and (iv) a memory communicatively coupled to the processor, the memory configured to store the program module, the program module including instructions for: identifying, by the wireless communication device, an intention gesture indicating an intention to control the object, wherein the object is positioned proximate to the wireless communication device; determining the object associated with the intention gesture based at least in part on wireless ranging; accessing sensor data generated by the one or more sensors; interpreting the accessed sensor data to identify an action gesture; and transmitting, using the interface circuit, a command value corresponding to the action gesture to control the object.

[0172] In some embodiments, the motion gesture comprises a composite gesture. In some embodiments, determining the object is also based at least in part on receiving an identifier associated with the object. In some embodiments, the program module further comprises instructions for initiating a pairing operation with the object.

[0173] In some embodiments, a computer program product for use in conjunction with a wireless communication device includes a non-transitory computer-readable storage medium and a computer program mechanism embedded therein for controlling an object, the computer program mechanism including: (i) instructions for identifying, by the wireless communication device, an intended gesture indicating an intention to control the object, wherein the object is positioned proximate to the wireless communication device; (ii) instructions for determining, by the wireless communication device, the object associated with the intended gesture based at least in part on wireless ranging; (iii) instructions for accessing, by the wireless communication device, sensor data generated by one or more sensors associated with the wireless communication device; (iv) instructions for interpreting the accessed sensor data to determine an action gesture; and (v) instructions for transmitting a command corresponding to the action gesture to control the object.

[0174] In some embodiments, the instructions for determining the object further include instructions for determining the orientation of the wireless communication device based at least in part on sensor data generated by the one or more sensors associated with the wireless communication device. In some embodiments, the instructions for determining the object further include instructions for ignoring another object based at least in part on the orientation of the wireless communication device and the wireless ranging.

[0175] In some embodiments, a method for locating an object implemented by a wireless communication device includes the wireless communication device: (i) receiving a transmission including an anonymous identifier associated with an object located proximate to the wireless communication device; (ii) determining a range and direction of the object relative to the wireless communication device, wherein the range and the direction are determined at least in part using wireless ranging operations; and (iii) presenting output information indicating the position of the object based at least in part on the range and the direction.

[0176] In some embodiments, presenting the output information includes: displaying a map of the proximity area on the display of the wireless communication device; and displaying an indicator on the map indicating the location of the object. In some embodiments, the indicator is displayed regardless of whether the object is within the visual range of the wireless communication device. In some embodiments, presenting the output information includes: displaying an image of the proximity area captured by the image sensor of the wireless communication device on the display of the wireless communication device; and displaying an indicator on the image indicating the location of the object. In some embodiments, the indicator is displayed regardless of whether the object is within the visual range of the wireless communication device. In some embodiments, the anonymous identifier associated with the object includes a transient identifier. In some embodiments, the anonymous identifier associated with the object identifies a resource class corresponding to the object. In some embodiments, the method further includes comparing the received anonymous identifier with one or more stored target identifiers to determine a match. In some embodiments, the one or more stored target identifiers are received together with the search results. In some embodiments, at least one of the one or more stored target identifiers corresponds to a contact stored at the wireless communication device. In some embodiments, presenting the output information further comprises: switching between a map showing a proximity area including a first indicator representing the object and an image captured by a camera showing the proximity area including a second indicator corresponding to the object based on the orientation of the wireless communication device. In some embodiments, the indicator is sized according to the range. In some embodiments, the method further comprises transmitting a message to the object using the anonymous identifier. In some embodiments, the message comprises an audio message transmitted as a beamformed audio message that is substantially perceptible only at a position corresponding to the object, wherein the position is determined at least in part based on the range and the direction. In some embodiments, the output information comprises a tactile output. In some embodiments, the tactile output is presented by a device associated with the wireless communication device.

[0177] In some embodiments, a wireless communication device includes: (i) an interface circuit configured to wirelessly communicate with an object using at least one wireless communication protocol; (ii) a processor communicatively coupled to the interface circuit, the processor configured to execute a program module; and (iii) a memory communicatively coupled to the processor, the memory configured to store the program module, the program module including instructions for: receiving, via the interface circuit, a transmission including an anonymous identifier associated with an object positioned proximate to the wireless communication device; determining, via the interface circuit, a range and a direction of the object relative to the wireless communication device, wherein the range and the direction are determined at least in part using a wireless ranging operation; and presenting output information indicating the range and the direction of the object relative to the wireless communication device.

[0178] In some embodiments, the instructions for presenting output information also include instructions for presenting a map including a location indicator representing the object.

[0179] In some embodiments, a computer program product for use in conjunction with a wireless communication device includes a non-transitory computer-readable storage medium and a computer program mechanism embedded therein for locating an object, the computer program mechanism including: (i) instructions for receiving, by the wireless communication device, a transmission including an anonymous identifier associated with an object located proximate to the wireless communication device; (ii) instructions for determining, by the wireless communication device, a range and a direction of the object relative to the wireless communication device, wherein the range and the direction are determined at least in part using wireless ranging operations; and (iii) instructions for presenting output information indicating the range and the direction of the object relative to the wireless communication device.

[0180] In some embodiments, the instructions for presenting also include instructions for transitioning between displaying a map of the proximity area including a first indicator representing the object and displaying an image captured by a camera of the proximity area including a second indicator corresponding to the object based on the orientation of the wireless communication device.

[0181] In the foregoing description, reference has been made to “some embodiments.” Note that “some embodiments” describes a subset of all possible embodiments, but does not always specify the same subset of embodiments.

[0182] The foregoing description is intended to enable any person skilled in the art to implement and use the present disclosure, and is provided in the context of a specific application and its requirements. In addition, the foregoing description of the embodiments of the present disclosure is presented for the purpose of illustration and description only. They are not intended to be exhaustive or to limit the present disclosure to the disclosed form. Therefore, many modifications and variations will be apparent to practitioners skilled in the art, and the general principles defined herein can be applied to other embodiments and applications without departing from the essence and scope of the present disclosure. In addition, the discussion of the foregoing embodiments is not intended to limit the present disclosure. Therefore, the present disclosure is not intended to be limited to the embodiments shown, but will be given the widest scope consistent with the principles and features disclosed herein.

Claims

1. A method for controlling an object by a wireless device, the method comprising: By the wireless device: interpreting the first set of data to detect an intention gesture indicating an intention to control the object, wherein the object is within a wireless range of the wireless device; performing a wireless ranging operation to identify an object to be controlled, wherein performing the wireless ranging operation includes receiving an anonymous identifier from the object to distinguish the object from other proximate objects, and the anonymous identifier can only be resolved by a device with which the anonymous identifier and the identification information item have been shared; interpreting a second set of data to determine a motion gesture indicative of a command for controlling the object identified by the wireless ranging operation; as well as A command value corresponding to the command is sent to control the object. 2 . The method of claim 1 , wherein interpreting the first set of data comprises measuring an orientation of the wireless device relative to the location of the object. 3 . The method of claim 1 , wherein interpreting the second set of data comprises determining movement of the wireless device along two or three dimensions.

4. The method according to claim 1, wherein: The first set of data includes a first output from at least one of the one or more sensors in the wireless device; and The second set of data includes a second output from at least one of the one or more sensors in the wireless device. 5 . The method of claim 1 , wherein the wireless ranging operation comprises transmitting a wireless signal to the object and / or receiving a wireless signal from the object.

6. The method of claim 1, wherein performing the wireless ranging operation comprises interpreting characteristics of received wireless signals to determine a distance and / or direction of the object relative to the wireless device. The method of claim 1 , wherein the object comprises a physical object.

8. The method of claim 1, wherein the object comprises a virtual object comprising a proxy for a physical object at a different location.

9. The method of claim 1, wherein controlling the object comprises associating the wireless device with the object.

10. The method of claim 1, wherein controlling the object comprises changing an operational setting of the object.

11. A wireless device comprising: an interface circuit configured to communicate wirelessly with a subject; one or more sensors configured to measure sensor data; a processor communicatively coupled to the interface circuit and the one or more sensors and configured to execute a program module for controlling the object; and a memory, the memory being communicatively coupled to the processor and configured to store the program module, wherein the program module includes instructions for: interpreting a first set of sensor data to detect an intention gesture indicating an intent to control the object, wherein the object is within a wireless range of the wireless device; performing a wireless ranging operation to identify the object, wherein performing the wireless ranging operation includes receiving an anonymous identifier from the object to distinguish the object from other proximate objects, and the anonymous identifier can only be resolved by a device with which the anonymous identifier and the identification information item have been shared; interpreting a second set of sensor data to determine a motion gesture indicative of a command for controlling the object identified by the wireless ranging operation; as well as A command value corresponding to the command is sent to control the object.

12. The wireless device of claim 11, wherein interpreting the first set of data comprises measuring an orientation of the wireless device relative to the location of the object.

13. The wireless device of claim 11, wherein interpreting the second set of data comprises determining movement of the wireless device in two or three dimensions.

14. The wireless device of claim 11, wherein: The first set of data includes a first output from at least one of the one or more sensors in the wireless device; and The second set of data includes a second output from at least one of the one or more sensors in the wireless device.

15. The wireless device of claim 11, wherein the wireless ranging operation comprises transmitting a wireless signal to the object and / or receiving a wireless signal from the object.

16. The wireless device of claim 11, wherein performing the wireless ranging operation comprises interpreting characteristics of received wireless signals to determine a distance and / or direction of the object relative to the wireless device.

17. The wireless device of claim 11, wherein the object comprises a physical object.

18. The wireless device of claim 11, wherein the object comprises a virtual object comprising a proxy for a physical object at a different location.

19. The wireless device of claim 11, wherein controlling the object comprises: associating the wireless device with the object; or Changes the operational settings of the object.

20. A computer program product for use in conjunction with a wireless device, the computer program product comprising a non-transitory computer readable storage medium and a computer program mechanism embedded therein for controlling an object, the computer program mechanism comprising: instructions for interpreting a first set of data to detect an intentional gesture indicating an intention to control the object, wherein the object is within a wireless range of the wireless device; instructions for performing a wireless ranging operation to identify the object, wherein performing the wireless ranging operation includes receiving an anonymous identifier from the object to distinguish the object from other proximate objects, and the anonymous identifier can only be resolved by a device with which the anonymous identifier and the identification information item have been shared; instructions for interpreting a second set of data to determine a motion gesture indicative of a command for controlling the object identified by the wireless ranging operation; as well as An instruction for sending a command value corresponding to the command to control the object.

Images