Indoor map generation using radio frequency detection

Abstract

Systems and techniques are disclosed for performing mapping using radio frequency (RF) sensing. For example, a server may obtain a first set of RF sensing data and orientation data corresponding to a first wireless device from a plurality of wireless devices. The first set of RF sensing data may be associated with at least one received waveform that is a reflection of a transmitted waveform from a first reflector. Based on the first set of RF sensing data, the orientation data, and the location data corresponding to the first wireless device, an indoor map may be generated that includes a reference to the reflector.

Description

Technical Field

[0001] This disclosure generally relates to indoor mapping. Aspects of this disclosure relate to systems and techniques for generating indoor maps using radio frequency (RF) sensing.

Background Art

[0002] Wireless electronic devices can provide various telecommunications services, as well as geolocation, mapping, and route discovery functions. For example, a portable electronic device can include software and hardware components that can determine the location of the device and indicate to a user the direction to a specific destination.

[0003] To implement various telecommunications functions, a wireless electronic device can include hardware and software components configured to transmit and receive radio frequency (RF) signals. For example, a mobile device can be configured to communicate particularly via Wi-Fi, 5G / New Radio (NR), Bluetooth (trademark), and / or Ultra Wide Band (UWB).

Summary of the Invention

Means for Solving the Problems

[0004] The following presents a simplified summary of one or more aspects disclosed herein. Accordingly, the following summary should not be considered an extensive overview of all contemplated aspects, nor should the following summary be regarded as identifying key or critical elements of all contemplated aspects or as delineating the scope of any particular aspect. Thus, the sole purpose of the following summary is to present some concepts related to one or more aspects of the mechanisms disclosed herein in a simplified form prior to the detailed description presented below.

[0005] Systems, methods, apparatus, and computer-readable media for generating indoor maps are disclosed. According to at least one example, a method for generating an indoor map is provided. This method may include the steps of: a server device receiving a first set of radio frequency (RF) sensing data and orientation data from a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector; and the server device generating an indoor map which includes a reference to the first reflector based on the first set of RF sensing data, orientation data, and position data corresponding to the first wireless device.

[0006] In another example, an apparatus for indoor mapping is provided, comprising memory and at least one processor (for example, configured in a circuit) commutatively coupled to the memory. The at least one processor is configured to receive a first set of radio frequency (RF) sensing data and orientation data from a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and to generate an indoor map which includes a reference to the first reflector based on the first set of RF sensing data, orientation data and position data corresponding to the first wireless device.

[0007] In another example, a non-temporary computer-readable medium is provided, which stores at least one instruction, and when at least one instruction is executed by one or more processors, causes one or more processors to receive a first set of radio frequency (RF) sensing data and orientation data corresponding to a first wireless device from a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and causes a server device to generate an indoor map which includes a reference to the first reflector based on the first set of RF sensing data, orientation data and position data corresponding to the first wireless device.

[0008] In another example, a device for performing indoor mapping is provided. The device includes means for receiving a first set of radio frequency (RF) sensing data and orientation data from a plurality of wireless devices corresponding to a first wireless device, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector; and means for generating an indoor map which includes a reference to the first reflector based on the first set of RF sensing data, orientation data and position data corresponding to the first wireless device.

[0009] Another example provides a method for performing indoor mapping. The method includes: transmitting a radio frequency (RF) signal by a wireless device; receiving a plurality of reflected RF signals by the wireless device, each of which is a reflection of a transmitted RF signal from at least one object in an indoor space; acquiring RF detection data for the plurality of reflected RF signals from at least one object by the wireless device; and presenting an indoor map of the indoor space by the wireless device, which includes references to at least one object, wherein the references to at least one object are based on the RF detection data.

[0010] In another example, an apparatus for indoor mapping is provided, comprising a transceiver, memory, and at least one processor (for example, configured in a circuit) coupled to the memory and the transceiver. The at least one processor is configured to transmit radio frequency (RF) signals via the transceiver, receive a plurality of reflected RF signals via the transceiver which are reflections of transmitted RF signals from at least one object in an indoor space, acquire RF sensing data for the plurality of reflected RF signals from at least one object, and present an indoor map of the indoor space which includes references to at least one object, wherein the references to at least one object are based on the RF sensing data.

[0011] In another example, a non-temporary computer-readable medium is provided, which stores at least one instruction, and when at least one instruction is executed by one or more processors, causes one or more processors to transmit a radio frequency (RF) signal, each receiving a plurality of reflected RF signals which are reflections of a transmitted RF signal from at least one object in an indoor space, to acquire RF detection data for the plurality of reflected RF signals from at least one object, and to present an indoor map of the indoor space which includes references to at least one object, wherein the references to at least one object are based on the RF detection data.

[0012] In another example, a device for performing indoor mapping is provided. The device includes means for transmitting radio frequency (RF) signals, means for receiving a plurality of reflected RF signals, each of which is a reflection of a transmitted RF signal from at least one object in an indoor space, means for acquiring RF detection data for the plurality of reflected RF signals from at least one object, and means for presenting an indoor map of the indoor space, including references to at least one object, wherein the references to at least one object are based on the RF detection data.

[0013] In some embodiments, the device is a mobile device (e.g., a mobile phone or so-called “smartphone” or other mobile device), a wearable device, an extended reality device (e.g., a virtual reality (VR) device, an augmented reality (AR) device, or a mixed reality (MR) device), a tablet, a personal computer, a laptop computer, a server computer, a wireless access point, or any other device having an RF interface, or a part thereof. In some embodiments, the device described above may include one or more sensors that can be used to determine the location of the device, the orientation of the device, and / or any other purpose.

[0014] Other purposes and advantages relating to the embodiments disclosed herein will become apparent to those skilled in the art based on the accompanying drawings and detailed description.

[0015] The accompanying drawings are provided to aid in describing various aspects of this disclosure and are provided solely for illustrative purposes of aspects, not as an limitation of those aspects. [Brief explanation of the drawing]

[0016] [Figure 1] This figure shows an example of a wireless communication network based on several embodiments. [Figure 2] A block of examples of computing systems for user devices, with several examples. [Figure 3] This figure shows examples of wireless devices that perform indoor mapping using radio frequency (RF) detection techniques, with several examples. [Figure 4] This figure shows examples of indoor environments, including wireless devices for detecting the presence of users and performing indoor mapping, with several examples. [Figure 5] This figure shows an example of a graphical representation of the size and location of an object detected based on RF detection techniques, based on several embodiments. [Figure 6]This flowchart illustrates an example of the process for creating indoor maps, using several examples. [Figure 7] This flowchart illustrates an example of the process for creating indoor maps, using several examples. [Figure 8] This block contains examples of computing systems, with several examples. [Modes for carrying out the invention]

[0017] Several aspects and embodiments of this disclosure are provided below for illustrative purposes. Alternative embodiments may be devised without departing the scope of this disclosure. Additionally, well-known elements of this disclosure are not described in detail or are omitted so as not to obscure the relevant details of this disclosure. As will be apparent to those skilled in the art, some of the aspects and embodiments described herein may be applied independently, and some may be applied in combination. Specific details are provided below for illustrative purposes to give a complete understanding of the embodiments of this application. However, it will be apparent that various embodiments may be practiced without these specific details. The figures and descriptions are not intended to be limiting.

[0018] The following description provides exemplary embodiments and is not intended to limit the scope, applicability, or configuration of the Disclosure. Rather, the following description of exemplary embodiments provides a description that enables the implementation of the exemplary embodiments for those skilled in the art. It should be understood that various modifications may be made to the function and configuration of the elements without departing from the spirit and scope of the Application as set forth in the appended claims.

[0019] Portable electronic devices such as smartphones, tablets, or laptops can execute functions that can include geolocation, mapping, and route discovery capabilities. Generally, these portable electronic devices include hardware and software components that utilize a global navigation satellite system (GNSS) such as the global positioning system (GPS) to determine the location of the device. Such systems provide a high degree of accuracy regarding the location of the device, but a direct line of sight between the device and the GNSS satellites is required to ensure proper functionality. Therefore, these systems are not suitable for indoor use and may be limited in some outdoor use cases.

[0020] To address the drawbacks associated with GNSS, techniques have been developed to facilitate indoor mapping by utilizing alternative mechanisms for tracking device location and / or device trajectories. For example, systems are available that utilize beacons to track the location of a device such as a robotic vacuum cleaner throughout an indoor space. By aggregating the location data, a map based on the locations accessible to the device can be generated. Thus, the system may assume that a location not reported by the device is inaccessible, for example, due to the presence of a wall or furniture.

[0021] Techniques also exist for tracking the trajectory of a mobile device throughout an indoor space by monitoring the device location at regular time intervals. For example, the movement or trajectory of the device can be mapped using location information and timestamp data associated with the device location. A map of the indoor environment can be generated using the accumulated device trajectory data.

[0022] These techniques can be used to render a basic two-dimensional map of an indoor space but cannot generate a three-dimensional map. Further, as described above, existing indoor map generation techniques rely on the position or trajectory of a mobile device and thereby only make estimates regarding the surrounding space using this information. Existing techniques do not, in particular, directly capture data regarding the environment surrounding the device, such as walls, floors, ceilings, doors, and furniture.

[0023] Furthermore, relying on device position to generate an indoor map is time-consuming because the device or the user holding the device needs to cover a large amount of space. Also, there may be some spaces that are accessible but not regularly visited, so there are inherent drawbacks to relying on position data. Thus, the resulting indoor map may contain a high level of inaccuracy regarding the objects within the indoor space as well as the overall layout.

[0024] This specification describes systems, apparatus, processes (also called methods), and computer-readable media (collectively referred to as "Systems and Techniques") for generating maps such as maps of indoor spaces (indoor maps), maps of outdoor spaces (outdoor maps), maps of spaces including both indoors and outdoors, other types of maps, or any combination thereof. This specification uses indoor maps as an example to illustrate embodiments. However, the Systems and Techniques are not limited to generating indoor maps and can be used to generate any type of map. The Systems and Techniques provide an electronic device with the ability to perform radio frequency (RF) sensing and acquire RF sensing data that can be used to generate maps of spaces (e.g., indoor spaces). RF sensing data can be acquired by utilizing a wireless interface capable of simultaneously performing transmitting and receiving functions. Hereinafter, Wi-Fi is used as an exemplary example to illustrate embodiments. However, the Systems and Techniques are not limited to Wi-Fi. For example, in some cases, the Systems and Techniques can be implemented using 5G / New Radio (NR), such as using millimeter wave (mm wave) technology. In some cases, the Systems and Techniques can be implemented using other wireless technologies such as Bluetooth®, ultra-wideband (UWB), etc.

[0025] In some examples, systems and techniques can perform RF sensing by implementing a Wi-Fi interface for a device having at least two antennas that can be used to simultaneously transmit and receive RF signals. In some cases, the antennas can be omnidirectional so that they can receive and transmit RF signals in all directions. For example, a device may transmit an RF signal using the transmitter of its Wi-Fi interface, and at the same time, enable the Wi-Fi receiver of the Wi-Fi interface so that the device can capture any signals reflected from reflectors (e.g., static or dynamic objects and / or structural elements) in the surrounding environment. The Wi-Fi receiver can also receive leak signals that are not reflected from objects and are directly coupled from the antenna of the Wi-Fi transmitter to the antenna of the Wi-Fi receiver. In doing so, the device may collect RF sensing data in the form of channel state information (CSI) based on data about the direct path of the transmitted signal (leak signal), along with data about the reflected path of the received signal corresponding to the transmitted signal.

[0026] In some embodiments, CSI data can be used to calculate the distance and angle of arrival of reflected signals. Using the distance and angle of reflected signals, the size and location of reflectors in the surrounding environment can be identified to generate an indoor map.

[0027] In some examples, the distance and angle of arrival of a reflected signal can be determined by signal processing, machine learning algorithms, any other suitable technique, or any combination thereof. In one example, the distance of the reflected signal can be calculated by measuring the time difference between receiving the leak signal and receiving the reflected signal. In another example, the angle of arrival can be calculated by using an antenna array to receive the reflected signal and measuring the difference in reception phase at each element of the antenna array.

[0028] In some embodiments, the distance of a reflected signal, along with its angle of arrival, can be used to determine the size and shape of the object causing the reflection. For example, a device can use the calculated distance and angle of arrival to determine the point from which the signal was reflected by an object. This allows the device to aggregate the reflection points for various signals to determine the size and shape of the reflector.

[0029] In some cases, a device may determine and store device position data and device orientation data. In some cases, device position data and device orientation data can be used to adjust calculations about the distance and angle of arrival (determined using CSI data) of reflected signals when the device is moving. For example, position and orientation data can be used to correlate one or more reflected signals with their corresponding transmitted signals. In some cases, device position (or location) data can be collected by using techniques to measure round-trip time (RTT), passive positioning, angle of arrival (AoA), received signal strength indicator (RSSI), using CSI data, using any other suitable technique, or any combination thereof. Device orientation data can be obtained from electronic sensors on the device, such as one or more gyroscopes, accelerometers, compasses, any other suitable sensors, or any combination thereof.

[0030] In some examples, the device itself may analyze RF sensing data (e.g., CSI data) and perform calculations of reflection distance and angle to identify objects / boundaries in the environment surrounding the device. The device may use this data to generate a local indoor map of itself. In some cases, the device may then upload its map information to a server that can retrieve map information from multiple devices. The server may aggregate the map data retrieved from multiple devices to further develop the indoor map. In some examples, the server may then provide the updated map information to the device.

[0031] In some cases, the device may upload RF detection data to a server. In some cases, the device may send its position and orientation data to the server along with the RF detection data. The server may perform calculations to determine the distance and angle of arrival of the reflected signal, thereby freeing the device from the computational overhead required to perform the calculations. Based on the determined distance and angle of arrival, the server may generate and provide an indoor map to the device.

[0032] In some examples, a server may develop a more detailed indoor map by "crowdsourcing" data from a large number of devices. Devices providing data to the server may or may not be associated with a local network (e.g., a Wi-Fi network) present in their respective indoor environments. For example, a server may receive data from devices located inside a building that provide local Wi-Fi, regardless of the device's association with a local network. In some examples, a wireless device may collect CSI data using an RF interface, such as a Wi-Fi transceiver, when it is not connected to a Wi-Fi router or Wi-Fi access point in the building. Similarly, a wireless device may collect RF sensing data using an RF interface when there is no available network in the indoor environment. The device may store the RF sensing data locally and send it to the server later when a network connection becomes available.

[0033] In some techniques, a device may provide or, in some cases utilize an absolute location to a server. For example, if a device is located inside a building but is close to an exterior wall or window, it may be possible to obtain a GPS fix, which allows the device to identify its absolute location. Alternatively, the device may utilize its last known GPS location, which can be used to estimate the location of the building from which the GPS fix was terminated. If an absolute location is known or estimated, that absolute location can be used to associate an interior map with a specific building. Alternatively, the device may utilize a relative location. For example, location data may relate to a Wi-Fi access point or Bluetooth® beacon located in the same environment as the device.

[0034] Various aspects of the techniques described herein are discussed below with respect to the drawings. Figure 1 is a block diagram of an exemplary communication network 100. According to some embodiments, the communication network 100 may include a wireless local area network (WLAN), such as a Wi-Fi network 108 (hereinafter referred to as WLAN 108). For example, WLAN 108 may be a network that implements at least one of the IEEE 802.11 wireless communication protocol standards (including, but not limited to, 802.11ay, 802.11ax, 802.11az, 802.11ba, and 802.11be, as defined by the IEEE 802.11-2016 specification or its amendments).

[0035] WLAN108 may include a number of wireless communication devices, such as access points (APs) 102 and user equipment (UEs) 104a, 104b, 104c, and 104d (collectively referred to as "UE 104"). Although only one AP 102 is illustrated, WLAN108 may also include multiple APs 102. Generally, a UE may be any wireless communication device used by a user to communicate over a wireless communication network (e.g., a mobile phone, router, tablet computer, laptop computer, wearable device (e.g., smartwatch, glasses, virtual reality (VR) headset, augmented reality (AR) headset or glasses, or extended reality (XR) device such as a mixed reality (MR) headset), vehicle (e.g., car, motorcycle, bicycle, etc.), Internet of Things (IoT) device, etc.). A UE may be mobile or stationary (e.g., at some point in time) and may communicate with a wireless access network (RAN). As used herein, the term “UE” may be interchangeably referred to as “Access Terminal” or “AT,” “User Device,” “User Terminal” or “UT,” “Client Device,” “Wireless Device,” “Subscriber Device,” “Subscriber Terminal,” “Subscriber Station,” “Mobile Device,” “Mobile Terminal,” “Mobile Station,” or variations thereof. Generally, a UE can communicate with the core network via the RAN, and through the core network, a UE can connect to external networks such as the Internet and to other UEs. A UE can also communicate with other UEs and / or other devices as described herein.

[0036] A single AP102 and an associated set of UE104 may be referred to as a Basic Service Set (BSS) managed by each AP102. The BSS may be identified to users by a Service Set Identifier (SSID) and to other devices by a Basic Service Set Identifier (BSSID), which may be the AP102's Media Access Control (MAC) address. The AP102 periodically broadcasts beacon frames ("beacons") containing the BSSID so that any UE104 within the AP102's wireless range can "associate" with or reassociate with the AP102 to establish or maintain their respective (hereinafter also called "Wi-Fi links") communication links 106 with the AP102. For example, a beacon may include identification information for the primary channel used by each AP102, as well as timing synchronization functionality for establishing or maintaining timing synchronization with the AP102. The AP102 may provide access to external networks to various UE104 in the WLAN via their respective communication links 106.

[0037] To establish a communication link 106 with AP102, each UE104 is configured to perform a passive scan operation or an active scan operation ("scan") on a frequency channel in one or more frequency bands (for example, a 2.4 GHz, 5 GHz, 6 GHz, or 60 GHz band). To perform a passive scan, the UE104 listens for beacons, which are transmitted by each AP102 at periodic time intervals called the Target Beacon Transmission Time (TBTT) (measured in units of time (TU), where 1 TU may be equal to 1024 microseconds (μs)). To perform an active scan, the UE104 generates probe requests, transmits them sequentially on each channel to be scanned, and listens for probe responses from AP102. Each UE104 may be configured to identify or select an AP102 to associate with based on the scan information obtained through the passive or active scan, and to perform authentication and association operations to establish a communication link 106 with the selected AP102. AP102 assigns an association identifier (AID) to UE104 during the peak of association activity, and AP102 uses the AID to track UE104.

[0038] As a result of the increased ubiquity of wireless networks, UE104 may have the opportunity to select one of many BSSs within the UE's range, or to select from several AP102s that together form an Extended Service Set (ESS) containing multiple connected BSSs. An Extended Network Station associated with WLAN108 may be connected to a wired or wireless distribution system that may allow multiple AP102s to be connected within such an ESS. Thus, UE104 can be covered by two or more AP102s and can be associated with different AP102s at different times for different transmissions. In addition, after association with an AP102, UE104 may also be configured to periodically scan its vicinity to find a more suitable AP102 to associate with. For example, UE104 moving toward an associated AP102 may perform a “roaming” scan to find another AP102 with more desirable network characteristics, such as a higher Received Signal Strength Indicator (RSSI) or lower traffic load.

[0039] In some cases, the UE104 may form a network without AP102 or any other equipment other than the UE104 itself. One example of such a network is an ad-hoc network (or wireless ad-hoc network). Ad-hoc networks are sometimes referred to as mesh networks or peer-to-peer (P2P) networks. In some cases, the ad-hoc network may be implemented within a larger wireless network such as WLAN108. In such an implementation, the UE104 may be able to communicate with each other through AP102 using communication link 106, but the UE104 can also communicate with each other directly via direct wireless link 110. Furthermore, two UE104 may communicate via one or more device-to-device (D2D) peer-to-peer (P2P) links called “sidelinks”. In the example in Figure 1, UE104b has a direct wireless link 110 (e.g., a D2D P2P link) with UE104a, and UE104a is connected to one or more base stations 160, enabling UE104b to indirectly obtain cellular connectivity. Although Figure 1 shows a single base station 160, the communication system 100 may include multiple base stations communicating with UE104. In one example, the direct wireless link 110 may be supported using any well-known D2D radio access technology (RAT), such as LTE Direct (LTE-D), WiFi Direct (WiFi-D), Bluetooth®, or UWB.

[0040] AP102 and UE104 may function and communicate (via their respective communication link 106) in accordance with the IEEE 802.11 wireless communication protocol standards (including, but not limited to, 802.11ay, 802.11ax, 802.11az, 802.11ba, and 802.11be, as defined by the IEEE 802.11-2016 specification or its amendments). These standards specify WLAN radio and baseband protocols for the PHY layer and the Media Access Control (MAC) layer. AP102 and UE104 transmit and receive wireless communication (hereinafter also referred to as "Wi-Fi communication") between themselves in the form of PHY protocol data units (PPDUs) (or Physical Layer Convergence Protocol (PLCP) PDUs). AP102 and UE104 within WLAN108 may transmit PPDUs over unlicensed spectrum, which may be a portion of the spectrum including frequency bands conventionally used by Wi-Fi technology, such as the 2.4GHz band, 5GHz band, 60GHz band, 3.6GHz band, and 900MHz band. Some implementations of AP102 and UE104 described herein may also communicate over other frequency bands, such as the 6GHz band, which may support both licensed and unlicensed communications. AP102 and UE104 may also be configured to communicate over other frequency bands, such as shared licensed frequency bands, where multiple operators may have licenses to operate in one or more of the same or overlapping frequency bands.

[0041] Each frequency band may include multiple subbands or frequency channels. For example, PPDUs compliant with IEEE 802.11n, 802.11ac, 802.11ax, and 802.11be standard amendments may be transmitted over a 2.4GHz, 5GHz, or 6GHz band, each divided into multiple 20MHz channels. Thus, these PPDUs are transmitted over physical channels with a minimum bandwidth of 20MHz, but larger channels can be formed through channel bonding. For example, a PPDU may be transmitted over physical channels with bandwidths of 40MHz, 80MHz, 160MHz, or 200MHz by bonding multiple 20MHz channels together.

[0042] In some examples, the communication network 100 may include one or more base stations 160. One or more base stations 160 may include macrocell base stations (high-power cellular base stations) and / or small cell base stations (low-power cellular base stations). In some embodiments, the macrocell base station may include an eNB and / or ng-eNB corresponding to a 4G / LTE network, or a gNB corresponding to a 5G / NR network, or a combination of both, and the small cell base station may include a femtocell, picocell, microcell, etc.

[0043] One or more base stations 160 may collectively form a radio access network (RAN) and interface with a core network 170 (e.g., an evolved packet core (EPC) or a 5G core (5GC)) via a backhaul link 122, and interface with one or more servers 172 (which may be part of or outside the core network 170) via the core network 170. In addition to other functions, one or more base stations 160 may perform functions related to one or more of the following: forwarding user data, radio channel encryption and decryption, integrity protection, header compression, mobility control functions (e.g., handover, dual connectivity), inter-cell interference coordination, connection setup and release, load balancing, distribution for non-access stratum (NAS) messages, NAS node selection, synchronization, RAN sharing, multimedia broadcast multicast service (MBMS), subscriber and equipment tracing, RAN information management (RIM), paging, positioning, and delivery of warning messages.

[0044] One or more base stations 160 may communicate wirelessly with UEs such as UE104a via a communication link 120. The communication link 120 between one or more base stations 160 and UE104 may include uplink (also called reverse link) transmissions from the UEs (e.g., UE104a, 104b, 104c, and / or 104d) to base station 160, and / or downlink (also called forward link) transmissions from base station 160 to one or more UE104. The communication link 120 may use MIMO antenna technology, including spatial multiplexing, beamforming, and / or transmit diversity. The communication link 120 may be via one or more carrier frequencies.

[0045] Each UE 104 communicating with the communication network 100 may be configured to perform an RF sensing function for generating an indoor map. The RF sensing function can be performed using any of the RF interfaces present on the UE 104 that can simultaneously send and receive RF signals. The UE 104 can transfer data related to indoor mapping (e.g., RF sensing data, submap data, location data, orientation data, etc.) by utilizing the communication network 100.

[0046] In some examples, UE104 can communicate with one or more servers, such as server 172, for the purpose of generating indoor maps. Communication with server 172 can be done via core network 170, which may be accessed by UE104 by utilizing a communication link with base station 160 or AP102. AP102 can access the core network, including server 172, via communication link 112.

[0047] Figure 2 shows an example of a computing system 270 for a user device 207. User device 207 is an example of a UE that can be used by an end user. For example, user device 207 may include a mobile phone, router, tablet computer, laptop computer, wearable device (e.g., smartwatch, glasses, XR device, etc.), Internet of Things (IoT) device, and / or other devices used by the user to communicate over a wireless communication network. Computing system 270 includes software and hardware components that may be electrically or communicatively coupled (or otherwise communicated as appropriate) via bus 289. For example, computing system 270 includes one or more processors 284. One or more processors 284 may include one or more CPUs, ASICs, FPGAs, APs, GPUs, VPUs, NSPs, microcontrollers, dedicated hardware, any combination thereof, and / or other processing devices or systems. Bus 289 may be used by one or more processors 284 for intercore and / or communication with one or more memory devices 286.

[0048] The computing system 270 may also include one or more memory devices 286, one or more digital signal processors (DSPs) 282, one or more subscriber identification modules (SIMs) 274, one or more modems 276, one or more wireless transceivers 278, one or more antennas 287, one or more input devices 272 (e.g., a camera, mouse, keyboard, touch-sensitive screen, touchpad, keypad, microphone, etc.), and one or more output devices 280 (e.g., a display, speaker, printer, etc.).

[0049] One or more wireless transceivers 278 can receive wireless signals (e.g., signals 288) via antenna 287 from one or more other devices, such as user devices, network devices (e.g., base stations such as eNBs and / or gNBs, Wi-Fi access points (APs) such as routers, range extenders, etc.), and cloud networks. In some examples, the computing system 270 may include multiple antennas or antenna arrays that can facilitate simultaneous transmission and reception functions. In some examples, antenna 287 may be an omnidirectional antenna that can receive and transmit RF signals in all directions. Wireless signals 288 may be transmitted over a wireless network. The wireless network may be any wireless network, such as a cellular or telecommunications network (e.g., 3G, 4G, 5G, etc.), a wireless local area network (e.g., a Wi-Fi network), a Bluetooth® network, and / or other networks. In some examples, one or more wireless transceivers 278 may include an RF front end that includes one or more components, among other things, an amplifier, a mixer (also called a signal multiplier) for signal down-conversion, a frequency combiner (also called an oscillator) that provides the signal to the mixer, a baseband filter, an analog-to-digital converter (ADC), and one or more power amplifiers. The RF front end can generally handle the selection of wireless signals 288 and their conversion to baseband frequencies or intermediate frequencies, and can convert RF signals into the digital domain.

[0050] In some cases, the computing system 270 may include an encoding-decoding device (or codec) configured to encode and / or decode data transmitted and / or received using one or more wireless transceivers 278. In some cases, the computing system 270 may include an encryption-decoding device or component configured to encrypt and / or decrypt data transmitted and / or received by one or more wireless transceivers 278 (for example, according to AES and / or DES standards).

[0051] Each of the one or more SIMs 274 can securely store an International Mobile Subscriber Identification (IMSI) number and associated key assigned to the user of the user device 207. The IMSI and key can be used to identify and authenticate the subscriber when accessing a network provided by a network service provider or operator associated with the one or more SIMs 274. One or more modems 276 can modulate one or more signals to encode information for transmission using one or more wireless transceivers 278. One or more modems 276 can also demodulate signals received by one or more wireless transceivers 278 to decode the transmitted information. In some examples, one or more modems 276 can include Wi-Fi modems, 4G (or LTE) modems, 5G (or NR) modems, and / or other types of modems. One or more modems 276 and one or more wireless transceivers 278 can be used to communicate data for one or more SIMs 274.

[0052] The computing system 270 may also include one or more non-temporary machine-readable storage media or storage devices (e.g., one or more memory devices 286) (and / or communicate with them), which may include local storage and / or network-accessible storage, disk drives, drive arrays, optical storage devices, programmable, flash-updatable RAM and / or ROM, and other solid-state storage devices. Such storage devices may be configured to implement any suitable data storage device, including, but not limited to, various file systems, database structures, and the like.

[0053] In various embodiments, the function may be one or more computer program products (e.g., instructions or code) stored in memory device 286 and executed by one or more processors 284 and / or one or more DSPs 282. The computing system 270 may also include software elements (e.g., located in one or more memory devices 286), including other code such as an operating system, device drivers, executable libraries, and / or one or more application programs, and one or more application programs may comprise computer programs that perform the functions provided by various embodiments and / or implement the methods described herein and / or constitute the system described herein.

[0054] As described above, this specification describes systems and techniques for performing mapping (e.g., indoor mapping and / or other types of mapping) using radio frequency (RF) sensing. Figure 3 shows an example of a wireless device 300 that uses RF sensing techniques to detect an object 302 and perform indoor mapping. In some examples, the wireless device 300 can be a cell phone, a wireless access point, or several other devices that include at least one RF interface.

[0055] In some embodiments, the wireless device 300 may include one or more components for transmitting RF signals. The wireless device 300 may include a digital-to-analog converter (DAC) 304 which can receive a digital signal or waveform (for example, from a microprocessor not shown) and convert it to an analog waveform. The analog signal, which is the output of the DAC 304, can be provided to an RF transmitter 306. The RF transmitter 306 may be a Wi-Fi transmitter, a 5G / NR transmitter, a Bluetooth® transmitter, or any other transmitter capable of transmitting RF signals.

[0056] The RF transmitter 306 can be coupled to one or more transmitting antennas, such as the TX antenna 312. In some examples, the TX antenna 312 can be an omnidirectional antenna capable of transmitting RF signals in all directions. For example, the TX antenna 312 can be an omnidirectional Wi-Fi antenna capable of radiating Wi-Fi signals (e.g., 2.4GHz, 5GHz, 6GHz, etc.) in a 360-degree radiation pattern. In another example, the TX antenna 312 can be a directional antenna that transmits RF signals in a specific direction.

[0057] In some examples, the wireless device 300 may also include one or more components for receiving RF signals. For example, the receiver lineup in the wireless device 300 may include one or more receiving antennas, such as an RX antenna 314. In some examples, the RX antenna 314 may be an omnidirectional antenna capable of receiving RF signals in multiple directions. In other examples, the RX antenna 314 may be a directional antenna configured to receive signals from a specific direction. In further examples, both the TX antenna 312 and the RX antenna 314 may include multiple antennas (e.g., elements) configured as an antenna array (e.g., a linear antenna array, a two-dimensional antenna array, a three-dimensional antenna array, or any combination thereof).

[0058] The wireless device 300 may also include an RF receiver 310 coupled to an RX antenna 314. The RF receiver 310 may include one or more hardware components for receiving RF waveforms such as Wi-Fi signals, Bluetooth® signals, 5G / NR signals, or any other RF signals. The output of the RF receiver 310 may be coupled to an analog-to-digital converter (ADC) 308. The ADC 308 may be configured to convert the received analog RF waveform into a digital waveform that can be provided to a processor such as a digital signal processor (not shown).

[0059] In one example, the wireless device 300 can implement an RF sensing technique by transmitting a TX waveform 316 from a TX antenna 312. Although the TX waveform 316 is illustrated as a single line, some techniques allow the TX waveform 316 to be transmitted in all directions by an omnidirectional TX antenna 312. In one example, the TX waveform 316 can be a Wi-Fi waveform transmitted by a Wi-Fi transmitter in the wireless device 300. In further examples, the TX waveform 316 can be implemented to have a sequence with full or near-full autocorrection characteristics. For example, the TX waveform 316 may include a single-carrier Zadoff sequence or symbols similar to orthogonal frequency division multiplexing (OFDM) long training field (LTF) symbols.

[0060] In some techniques, the wireless device 300 can further implement RF sensing techniques by performing transmit and receive functions simultaneously. For example, the wireless device 300 can enable its RF receiver 310 to receive the TX waveform 316 at the same time as or nearly simultaneously with the RF transmitter 306 transmitting it. In some examples, the transmission of a sequence or pattern contained in the TX waveform 316 can be repeated continuously so that the sequence is transmitted a certain number of times or over a certain duration. In some examples, the repetition of a pattern in the transmission of the TX waveform 316 can be used to avoid the loss of reception of reflected signals when the RF receiver 310 is made available after the RF transmitter 306. In one implementation example, the TX waveform 316 may include a sequence having a sequence length L that is transmitted two or more times, thereby making the RF receiver 310 available at a time of less than L after the start of transmission of the TX waveform 316 to receive reflections corresponding to the entire sequence without loss of information.

[0061] The wireless device 300 can receive signals corresponding to the TX waveform 316 by implementing simultaneous transmit and receive functions. For example, the wireless device 300 can receive signals reflected from reflectors within the range of the TX waveform 316, such as the RX waveform 318 reflected from object 302. The wireless device 300 can also receive leak signals (e.g., TX leak signal 320) that are directly coupled from the TX antenna 312 to the RX antenna 314 without being reflected from an object. In some techniques, the RX waveform 318 may contain multiple sequences corresponding to multiple copies of the sequence contained in the TX waveform 316. In some examples, the wireless device 300 can combine multiple sequences received by the RF receiver 310 to improve the signal-to-noise ratio (SNR).

[0062] The wireless device 300 can further implement RF sensing techniques by acquiring RF sensing data associated with each of the received signals corresponding to the TX waveform 316. In some examples, the RF sensing data may include channel state information (CSI) based on data about the direct path of the TX waveform 316 (e.g., leak signal 320), along with data about the reflected path corresponding to the TX waveform 316 (e.g., RX waveform 318).

[0063] In some techniques, RF sensing data (e.g., CSI data) can include information that can be used to determine how an RF signal (e.g., TX waveform 316) propagates from the RF transmitter 306 to the RF receiver 310. RF sensing data can include data corresponding to the effects on the transmitted RF signal due to multi-path propagation, scattering, fading, and power attenuation due to distance, or any combination thereof. In some examples, RF sensing data can include imaginary and real data (e.g., I / Q components) corresponding to each tone in the frequency domain across a specific bandwidth.

[0064] In some examples, RF sensing data can be used to calculate the distance and angle of arrival corresponding to reflected waveforms, such as RX waveform 318. In further examples, RF sensing data can also be used to detect motion, determine location, detect changes in location or motion patterns, obtain channel estimates, or any combination thereof. In some cases, the distance and angle of arrival of reflected signals can be used to identify the size and location of reflectors (e.g., object 302) in the surrounding environment to generate an indoor map. In some embodiments, RF sensing data can also be used to identify transient objects (e.g., people or pets walking in the indoor environment) that can be omitted from the indoor map.

[0065] The wireless device 300 can calculate the distance and angle of arrival corresponding to a reflected waveform (for example, the distance and angle of arrival corresponding to RX waveform 318) by signal processing, machine learning algorithms, using any other suitable technique, or any combination thereof. In another example, the wireless device 300 can transmit RF sensing data to another computing device, such as a server, which can perform calculations to obtain the distance and angle of arrival corresponding to RX waveform 318 or other reflected waveforms.

[0066] For example, the distance of the RX waveform 318 can be calculated by measuring the time difference between receiving the leak signal and receiving the reflected signal. For instance, the wireless device 300 can determine a zero baseline distance based on the difference (e.g., propagation delay) between the time the wireless device 300 transmits the TX waveform 316 and the time it receives the leak signal 320. The wireless device 300 can then determine the distance associated with the RX waveform 318 based on the difference between the time the wireless device 300 transmits the TX waveform 316 and the time it receives the RX waveform 318, and can adjust this difference according to the propagation delay associated with the leak signal 320. In doing so, the wireless device 300 can determine the distance the RX waveform 318 has traveled, which can then be used to determine the distance of the reflector (e.g., object 302) that caused the reflection.

[0067] In further examples, the angle of arrival of the RX waveform 318 can be calculated by measuring the difference in arrival times of the RX waveform 318 between individual elements of a receiving antenna array, such as antenna 314. In some examples, the difference in arrival times can be calculated by measuring the difference in received phase at each element in the receiving antenna array.

[0068] Several techniques allow the distance and angle of arrival of the RX waveform 318 to be used to determine the distance between the wireless device 300 and the object 302, as well as the position of the object 302 relative to the wireless device 300. The distance and angle of arrival of the RX waveform 318 can also be used to determine the size and shape of the object 302 that causes the reflection. For example, the wireless device 300 can use the calculated distance and angle of arrival corresponding to the RX waveform 318 to determine the point where the TX waveform 316 was reflected from the object 302. The wireless device 300 can then aggregate the reflection points for various reflected signals to determine the size and shape of the object 302.

[0069] As described above, the wireless device 300 can include mobile devices such as smartphones, laptops, and tablets. In some examples, the wireless device 300 can be configured to acquire device position data and device orientation data along with RF sensing data. In some cases, the device position data and device orientation data can be used to determine or adjust the distance and angle of arrival of a reflected signal, such as an RX waveform 318. For example, a user may walk around a room holding the wireless device 300 during the RF sensing process. In this example, the wireless device 300 may have a first position and a first orientation when transmitting a TX waveform 316, and a second position and a second orientation when receiving an RX waveform 318. The wireless device 300 can take into account changes in position and orientation when processing the RF sensing data to calculate distance and angle of arrival. For example, the position data, orientation data, and RF sensing data can be correlated based on a timestamp associated with each element of the data. In some techniques, a combination of position data, orientation data, and RF sensing data can be used to determine the size and position of an object 302.

[0070] In some examples, device position data can be collected by the wireless device 300 using techniques including round-trip time (RTT) measurements, passive positioning, angle of arrival, received signal strength indicator (RSSI), CSI data, any other suitable technique, or any combination thereof. In further examples, device orientation data can be obtained from electronic sensors on the wireless device 300, such as a gyroscope, accelerometer, compass, magnetometer, any other suitable sensor, or any combination thereof. For example, a gyroscope on the wireless device 300 can be used to detect or measure changes in the orientation of the wireless device 300 (e.g., relative orientation), and a compass can be used to detect or measure the absolute orientation of the wireless device 300.

[0071] Figure 4 shows an indoor environment 400 which may include one or more wireless devices configured to perform RF sensing and create an indoor map. In some examples, the indoor environment 400 may include one or more mobile wireless devices (e.g., mobile device 402) and / or one or more fixed wireless devices (e.g., access point (AP) 404) which can be configured to perform RF sensing data and create an indoor map of the indoor environment 400.

[0072] In one embodiment, AP404 may be a Wi-Fi access point having a stationary or fixed position within an indoor environment 400. Although the indoor environment 400 is illustrated to have an access point (e.g., AP404), any type of fixed wireless device (e.g., a desktop computer, wireless printer, camera, smart television, smart home appliance, etc.) may be configured to perform the techniques described herein. In one example, AP404 may include hardware and software components that can be configured to simultaneously transmit and receive RF signals, such as the components described herein with respect to wireless device 300. For example, AP404 may include one or more antennas that can be configured to transmit RF signals (e.g., TX antenna 406) and one or more antennas that can be configured to receive RF signals (e.g., RX antenna 408). As noted with respect to wireless device 300, AP404 may include an omnidirectional antenna or antenna array configured to transmit and receive signals from any direction.

[0073] In one embodiment, AP404 can transmit an RF signal 410 that can be reflected from various reflectors located within the indoor environment 400 (e.g., static or dynamic objects located in the scene, structural elements such as walls, ceilings, or other barriers, and / or other objects). For example, the RF signal 410 can be reflected from wall 422, and the reflected signal 412 can be received by AP404 via RX antenna 408. When transmitting the RF signal 410, AP404 can also receive a leak signal 414 corresponding to the direct path from TX antenna 406 to RX antenna 408.

[0074] In some techniques, the AP404 can acquire RF sensing data associated with the reflected signal 412. For example, the RF sensing data may include CSI data corresponding to the reflected signal 412. In a further embodiment, the AP404 can use the RF sensing data to calculate the distance D1 and angle of arrival θ1 corresponding to the reflected signal 412. For example, the AP404 can determine the distance D1 by calculating the time of flight for the reflected signal 412 based on the difference or phase shift between the leak signal 414 and the reflected signal 412. In a further example, the AP404 can determine the angle of arrival θ1 by utilizing an antenna array to receive the reflected signal and measuring the difference in received phase at each element of the antenna array.

[0075] In some techniques, the AP404 can identify a wall 422 using a distance D1 and an angle of arrival θ1 corresponding to one or more reflected signals (e.g., reflected signal 412). In some embodiments, the AP404 can generate a map of the indoor environment 400 that includes references to the wall 422. In further embodiments, the AP404 can communicate with a server (e.g., server 172) to provide data for modifying the map of the indoor environment 400 to include references to the wall 422. In further examples, the AP404 can collect and provide RF detection data to a server for processing calculations of time of flight and angle of arrival for reflected signals.

[0076] In a further example, the indoor environment 400 may also include a mobile device 402. While illustrated as a smartphone, the mobile device 402 could include any type of mobile device, such as a tablet, laptop, or smartwatch. According to several techniques, the mobile device 402 can be configured to perform RF sensing to create or modify an indoor map of the indoor environment 400.

[0077] In one example, the mobile device 402 can transmit an RF waveform 416a via one of its RF transmitters, such as RF transmitter 306. As shown in the figure, the RF waveform 416a is transmitted at position (0,0) at time t=0. In some examples, the mobile device 402 can move while performing RF sensing to be located at a different position at a later time. This is shown as (x,y) at time t=t1.

[0078] In another example, the RF waveform 416a is reflected from object 420, and the reflected waveform 418a can be received by the mobile device 402 at time t=t1. In a further example, the wavelength of the RF waveform 416a can be configured to allow the RF waveform 416a to penetrate and / or cross object 420 and be reflected from wall 424. The reflection 418b from wall 424 can similarly cross object 420, thereby a second reflected waveform 418c can be received by the mobile device 402 at a later time, for example, t=t2 (not shown).

[0079] In some techniques, the mobile device 402 can collect RF sensing data corresponding to reflected waveforms 418a and 418c. In a further embodiment, the mobile device 402 can also capture device position data and device orientation data corresponding to the time when the RF waveform 416a was transmitted (e.g., t=0), and the time when the reflected waveforms 418a (e.g., t=t1) and 418c (e.g., t=t2) were received.

[0080] In some embodiments, the mobile device 402 can use RF detection data to calculate the time of flight and angle of arrival for each reflected waveform 418a and 418c. In further examples, the mobile device 402 can use position and orientation data to account for the device's movement during the RF detection process. For example, the time of flight of the reflected waveforms 418a and 418c can be adjusted based on the device's movement toward object 420 and wall 424, respectively. In another example, the angle of arrival of the reflected waveforms 418a and 418c can be adjusted based on the time the mobile device transmitted RF waveform 416a and the time the mobile device 402 received the reflected waveforms 418a and 418c, and the movement and orientation of the mobile device at that time.

[0081] In some techniques, the mobile device 402 can use time-of-flight, angle of arrival, position data, and orientation data to determine the size and location of object 420 and wall 424. Figure 5 is an example of a graphical representation 500 showing the size and location of object 420 and wall 424 based on RF detection that can be performed by the mobile device 402.

[0082] As shown in the figure, the graphical representation 500 may include the angle of arrival in degrees on the x-axis and the distance in centimeters on the y-axis. The graphical representation 500 may further include references to object 420 and wall 424 based on the angle of arrival and time of flight of the reflected signal. The graphical representation 500 shows that reflections from reflectors positioned one behind the other can be detected using an RF sensing technique. In this example, an RF waveform 416a generates a first reflection from object 420 and a second reflection from wall 424, and these reflections are received by the mobile device 402.

[0083] As described with respect to AP404, the mobile device 402 can identify the size and shape of objects 420 and walls 424 using distance, angle of arrival, position data, and orientation data. In some techniques, the mobile device 402 can use distance, angle of arrival, position, and orientation data to create a map of the indoor environment 400, including references to objects 420 and walls 424. In other techniques, the mobile device 402 can use RF sensing data to modify a partial map received from a server, such as server 172. In other embodiments, the mobile device 402 can send RF sensing data to a server to process and create an indoor map of the indoor environment 400.

[0084] In some examples, the AP404 and mobile device 402 can be configured to implement a bistatic configuration in which the transmit and receive functions are performed by different devices, respectively. For example, the AP404 (and / or other devices in a static or fixed indoor environment 400) can transmit an omnidirectional RF signal that may include signals 415a and 415b. As shown in the figure, signal 415a can be transmitted directly (e.g., without reflection) from the AP404 to the mobile device 402. Signal 415b can be reflected off the wall 426, and the corresponding reflected signal 415c can be received by the mobile device 402.

[0085] In some cases, the mobile device 402 can use RF sensing data associated with the direct signal path (e.g., signal 415a) and the reflected signal path (e.g., signal 415c) to identify the size and shape of a reflector (e.g., wall 426). For example, the wireless device 402 can acquire, retrieve, and / or estimate location data associated with AP 404. In some embodiments, the mobile device 402 can use location data and RF sensing data (e.g., CSI data) associated with AP 404 to determine time-of-flight, distance, and / or angle-of-arrival related signals (e.g., direct path signals such as signal 415a and reflected path signals such as signal 415c) transmitted by AP 404. In some cases, the mobile device 402 and AP 404 can further transmit and / or receive communications that may include data associated with RF signals 415a and / or reflected signals 415c (e.g., transmission time, sequence / pattern, arrival time, time-of-flight, angle-of-arrival, etc.).

[0086] In some examples, the mobile device 402 and / or AP404 can acquire RF sensing data in the form of CSI data, which can be used to formulate a matrix based on the number of frequencies represented as "K" (e.g., tones) and the number of receiving antenna array elements represented as "N". In one technique, the CSI matrix can be formulated according to the relationship given by equation (1). CSI matrix: H = [h ik ], i = 1,…, N, k = 1,…, K (1)

[0087] Mobile devices 402 and / or AP404 can calculate the angle of arrival and time of flight for both direct and reflected signal paths by formulating the CSI matrix and utilizing the two-dimensional Fourier transform. For example, the Fourier transform can be defined by the relationship given by equation (2) below, where K corresponds to the number of tones in the frequency domain, N corresponds to the number of receiving antennas, and hik Δf corresponds to the CSI data captured on the i-th antenna and the k-th tone (e.g., a complex number with real and imaginary parts), f0 corresponds to the carrier frequency, l corresponds to the antenna spacing, c corresponds to the speed of light, and Δf corresponds to the frequency spacing between two adjacent tones. The relationship in equation (2) is given as follows:

[0088]

number

[0089] In some techniques, the mobile device 402 and AP404 can perform RF sensing techniques independently of their relationship to each other or to a Wi-Fi network. For example, the mobile device 402 can utilize its Wi-Fi transmitter and Wi-Fi receiver to perform RF sensing as described herein when it is not associated with an access point or Wi-Fi network. In further examples, AP404 can perform RF sensing techniques regardless of whether a wireless device is associated with it.

[0090] In some embodiments, the mobile device 402 and AP404 can exchange data on their respective indoor maps of the indoor environment 400 to create a map that includes references to all reflectors (e.g., static objects, dynamic objects, structural elements) detected by both the mobile device 402 and AP404. In other embodiments, RF detection data from the mobile device 402 and AP404 can be transmitted to one or more servers that can aggregate the data to generate or modify the indoor map.

[0091] In some embodiments, a server device can acquire RF sensing data from multiple wireless devices located within an indoor environment (e.g., indoor environment 400) (e.g., crowdsourced). The server device can use the RF sensing data from multiple devices to identify and classify different reflectors. For example, the server device may use the RF sensing data to track the movement of an object, or determine that a reflector is a transient object (e.g., a pet or human walking in the environment) by determining that the data corresponding to the object is transient data and / or not confirmed by RF sensing data from other wireless devices. In some embodiments, the server device can omit and / or remove references to transient objects from the indoor map.

[0092] In another example, a server device may use RF sensing data from multiple wireless devices to determine that a reflector corresponds to a structural element such as a door, window, wall, floor, ceiling, roof, column, staircase, or any combination thereof. In some embodiments, the server device may include references in the indoor map indicating the type of structural element. In some cases, a server device may use RF sensing data from multiple wireless devices to determine that a reflector corresponds to a static object such as furniture, equipment, or fixtures (e.g., blinds / shades, ceiling fans, plants, carpets, lamps, etc.). In some embodiments, the server device may include references in the indoor map indicating the type of static object.

[0093] In some embodiments, a server device may use RF sensing data from multiple wireless devices to determine multiple position measurements corresponding to one or more reflectors. For example, two or more wireless devices may provide the server with RF sensing data corresponding to the same reflector. In some embodiments, when RF sensing data is received from different wireless devices, each corresponding to the same reflector, different position measurements may be obtained due to factors such as measurement errors, environmental changes, variations in signal propagation delay, semiconductor process variations, device configuration, any other factors that may affect RF sensing, and / or any combination thereof. In some embodiments, a server device may perform statistical analysis on the RF sensing data to determine the reflector's position. For example, a server device may determine the reflector's position by using the RF sensing data to calculate the mean, median, mode, standard deviation, range, standard score, any other statistical parameters, and / or any combination thereof. In some embodiments, a server device may assign weights and / or values ​​to the RF sensing data and / or measurements received from different wireless devices. In some embodiments, a server device may filter and / or discard RF sensing data from one or more wireless devices based on statistical analysis, processing, artificial intelligence, and / or machine learning algorithms.

[0094] In some examples, the server device may correlate or align RF sensing data received from wireless devices using location and / or orientation data corresponding to multiple wireless devices. For example, the server device may correlate RF sensing data from AP404 and mobile device 402 as data corresponding to object 420 based on location and / or orientation data associated with AP404 and mobile device 402. In some embodiments, the server device may use an algorithm for processing RF sensing data to interpolate and / or extrapolate the location, shape, size, and / or presence of one or more reflectors in an indoor environment. For example, RF sensing data received from multiple wireless devices can be processed by the server device to determine boundaries (e.g., walls, ceilings, floors, etc.) associated with a particular indoor environment or venue.

[0095] Figure 6 is a flowchart illustrating an example of process 600 for performing indoor mapping. Process 600 includes, in operation 602, receiving a first set of RF sensing data and orientation data from a plurality of wireless devices corresponding to a first wireless device. The first set of RF sensing data is associated with at least one received waveform, which is a reflection of the transmitted waveform of the first reflector. In some examples, the transmitted waveform may include signals that can be transmitted by an antenna from a wireless device such as device 207 (e.g., Wi-Fi signals, New Radio (NR) signals, RADAR signals, Bluetooth®, and / or ultra-wideband (UWB) signals). In further examples, the first set of RF sensing data may include CSI data corresponding to a reflection received in response to the transmission of a signal. In one exemplary example, the first set of RF sensing data may include Wi-Fi CSI data corresponding to a reflection received in response to the transmission of a Wi-Fi signal. In other examples, the first set of RF sensing data may include CSI data acquired using 5G NR, Bluetooth®, UWB, 60GHz mmWave, any combination thereof, or other types of signals.

[0096] In some implementations, a first set of RF sensing data and orientation data can be received by a server device from a wireless device (e.g., a user equipment (UE), a station (STA), or other device) having an RF interface configured to perform RF sensing. The wireless device can transmit the RF sensing data to the server device to avoid the computational overhead associated with processing the RF sensing data. In some examples, the server device can receive RF sensing data from multiple wireless devices located within the same indoor environment and aggregate the data to generate or develop a more comprehensive indoor map of the indoor environment. In some cases, the indoor map can be provided to the wireless device (e.g., from the server device). In some examples, the server device may send a communication to the wireless device requesting RF sensing data for indoor mapping (e.g., the server may determine that the wireless device is located within a room or building for which the server is creating or maintaining an indoor map).

[0097] In some embodiments, a server device can use RF sensing data from multiple wireless devices to identify one or more reflectors in an indoor environment (e.g., transient objects, static objects, structural elements, etc.). For example, in some cases, the server device may receive multiple RF sensing datasets corresponding to multiple wireless devices, and the multiple RF sensing datasets are associated with multiple received waveforms, which are reflections of at least one transmitted waveform from a first reflector. In some embodiments, the server can determine multiple position measurements associated with the first reflector based on the multiple RF sensing datasets. In some cases, the server can determine the position of the first reflector based on the multiple position measurements. In some examples, the position can be determined by performing a statistical analysis on the multiple position measurements (e.g., calculating the mean, average, median, standard deviation, etc.).

[0098] In some cases, the server device can use multiple RF sensing datasets from multiple wireless devices to determine that the first reflector corresponds to a transient object (e.g., a person, pet, robot, etc.). In response to determining that the first reflector corresponds to a transient object, the server device can remove and / or possibly omit the reference to the first reflector from the indoor map. In some examples, the server device can use multiple RF sensing datasets corresponding to multiple wireless devices to determine that the first reflector corresponds to a structural element. In some embodiments, the reference to the first reflector may indicate the type of structural element (e.g., a door, window, wall, floor, ceiling, roof, column, or a combination thereof). In some cases, the server device can use multiple RF sensing datasets corresponding to multiple wireless devices to determine that the first reflector corresponds to a static object. In some examples, the reference to the first reflector may indicate the type of static object (e.g., furniture, equipment, fixtures, or a combination thereof).

[0099] In some embodiments, the server can determine the distance and angle of arrival between a wireless device and a first reflector based on a first set of RF sensing data. In some examples, the distance determination may be based on the time of flight of the reflected signal adjusted for the propagation delay of the direct path (e.g., leakage signal between the transmitting and receiving antennas). In some examples, the angle of arrival may be based on the difference in signal phase measured at each element in the receiving antenna array. In some examples, the first set of RF sensing data may include the distance and angle of arrival between a first wireless device and a first reflector. For example, the wireless device may calculate the distance and angle of arrival and transmit the results of the calculation to the server.

[0100] Process 600 includes, in operation 604, generating an indoor map that includes a reference to a first reflector based on a first set of RF sensing data, orientation data, and position data corresponding to a first wireless device. For example, in some embodiments, a server device can generate an indoor map based on a first set of RF sensing data, orientation data, and position data. In some examples, position data can be derived from RF sensing data. In other examples, position data can be obtained using techniques that measure round-trip time (RTT), passive positioning, angle of arrival (AoA), received signal strength indicator (RSSI), or any combination thereof.

[0101] In further examples, location data can include relative or absolute location. For example, location data could be a relative location within a building, without a reference to the building's location on a map. In other examples, location data may include absolute location. For example, a wireless device located inside a building (e.g., a user equipment (UE), a station (STA), or another device) may obtain a GPS fix when positioned near a window, on a balcony, or any other location that facilitates line-of-sight communication with GPS satellites. In this example, the absolute location of the device can be used to associate an indoor map with an absolute location (e.g., an address on a map, GPS coordinates).

[0102] In some techniques, device orientation data can be obtained from one or more sensors on the mobile device, such as one or more accelerometers, compasses, and / or gyroscopes. For example, a gyroscope can be used to estimate changes in the mobile device's orientation (e.g., relative orientation), and a compass can be used to estimate the mobile device's absolute orientation. In some cases, both gyroscope and compass measurements can be used to determine the mobile device's absolute orientation. In some implementations, position and orientation data can be correlated with distance and angle of arrival based on timestamps associated with each other. The correlated data can be used to determine the size and / or location of the object that caused the signal reflection. An indoor map can then be generated or modified to include references to the object.

[0103] As described with respect to mobile device 402 in Figure 4, device movement during RF sensing (e.g., change in position or orientation) can affect how the reflected waveform is received. In some examples, a wireless device or server device may determine that the wireless device has moved during the RF sensing process. Accordingly, the distance and angle of arrival can be adjusted to account for the movement. The adjustment can be made according to wireless device position / location data and orientation data acquired simultaneously with or approximately simultaneously with the reception of the reflected waveform. Unless otherwise specified, position data and location data as used herein may refer to position and / or change in position / location. Orientation data may refer to orientation and / or change in orientation.

[0104] In some embodiments, as described with respect to Figure 4, RF sensing can be performed by two or more wireless devices (e.g., mobile device 402 and AP404). For example, RF sensing data corresponding to a second wireless device can be used to determine a second distance and a second angle of arrival between the second wireless device and a second object. Referring to Figure 4, mobile device 402 may correspond to a first wireless device that determines the distance and angle of arrival of a reflected signal from object 420 (e.g., the first object), and AP404 may correspond to a second wireless device that determines the distance and angle of arrival of a reflected signal from wall 422 (e.g., the second object). In some examples, the first wireless device (e.g., mobile device 402) may not be associated with a second wireless device (e.g., AP404). In other examples, the first wireless device may be associated with a second wireless device (e.g., AP404 may provide Wi-Fi connectivity with mobile device 402).

[0105] In some examples, RF sensing data corresponding to a first wireless device may overlap with RF sensing data corresponding to a second wireless device and may include reflections from at least one common object. Referring to Figure 4, mobile device 402 and AP404 can determine the distance and angle of arrival of a reflected signal from object 420 (reflection from AP404 is not shown). In some embodiments, corresponding indoor maps generated from each set of RF sensing data may overlap insofar as each map can include a reference to at least one common object. In some cases, the overlap between RF sensing data and / or corresponding indoor maps may be used by a server device to establish spatial relationships between sets of RF sensing data and / or indoor maps. Based on the spatial relationships, a combined indoor map may be generated that includes references to all objects present in both sets of RF sensing data (e.g., data from each wireless device). In some configurations, the server device can combine data based on RF sensing data, indoor maps (e.g., using a stitching process), references identified in the indoor maps, or any combination thereof. In some embodiments, a combined indoor map may be generated without knowledge of the respective positions and orientations of the first and second wireless devices. In some cases, location and / or orientation data can be used by a server device to support the generation of a combined indoor map by selectively combining RF sensing data and / or indoor maps from each wireless device based on the spatial relationships of the wireless devices (e.g., located in the same room and / or generally oriented in the same direction).

[0106] In response to a determination that the first and second wireless devices are located within the same indoor environment (e.g., indoor environment 400), the indoor map can be modified to include a second reference to the second object. In some embodiments, the first and second wireless devices can provide data (e.g., RF detection data, object references, etc.) to a server that can modify the indoor map. In some examples, the server can aggregate data from multiple wireless devices to generate or develop a more comprehensive indoor map of the indoor environment. In some cases, the first and second wireless devices can exchange data, and each wireless device may update its own copy of the indoor map.

[0107] In some embodiments, RF sensing can be performed using a bistatic configuration in which a first wireless device transmits a signal that is received by a second wireless device. For example, a Wi-Fi access point or some other wireless device can be configured to transmit a transmit waveform that corresponds to at least one received waveform.

[0108] Figure 7 is a flowchart illustrating an example of a process 700 for performing indoor mapping. Process 700 includes transmitting an RF signal in operation 702. Transmitting an RF waveform can be done using any suitable RF interface on a wireless device (e.g., a user equipment (UE), a station (STA), or other device), such as a wireless transceiver 278 on device 207. In some examples, the transmitted RF signal may correspond to a Wi-Fi signal that can be transmitted using one or more omnidirectional antennas (e.g., antenna 287). Process 700 includes receiving a plurality of reflected RF signals in operation 704, each of which is a reflection of the transmitted RF signal from at least one object in the indoor space. In some techniques, a plurality of reflected RF signals are received because the transmitted signal is radiated using an omnidirectional antenna facing different reflectors, each located in a different direction. In a further embodiment, a plurality of reflected RF signals are received because the transmitted signal is radiated and reflected through and / or across an object, causing a secondary reflection from an object located behind the first reflector. RF detection data can be used to identify the reflector associated with each received reflected signal.

[0109] Process 700 includes, in operation 706, acquiring RF sensing data for multiple reflected signals from at least one object. The RF sensing data is captured by using components such as those described with respect to device 300. For example, the RX antenna 314 can be used to receive reflected signals propagating to the ADC 308 via the RF receiver 310, and digital samples corresponding to the reflected signals can be captured. In some techniques, the RF sensing data can include both real and imaginary components corresponding to each frequency across a particular bandwidth. In one example, the RF sensing data can include CSI data.

[0110] Process 700 includes, in operation 708, presenting an indoor map of an indoor space, which includes a reference to at least one object, and the reference to at least one object is based on RF sensing data. In some embodiments, the reference to at least one object may include data indicating the location and / or extent of an object on the indoor map. In one example, a mobile device such as a smartphone may present the indoor map using an output device 280 such as a display. In another example, presenting the indoor map may include transferring the map to a different device, which may include a display. For example, an access point may present an indoor map by communicating with a mobile device within its range.

[0111] In some embodiments, a wireless device that transmits and receives RF signals can also calculate the distance and angle of arrival of a reflected signal. The calculated distance and angle of arrival may be based on RF detection data, device position data, and / or device orientation data. In some examples, the device position data may be based on RF detection data. In some cases, device orientation data may be obtained from one or more sensors on the mobile device, such as one or more accelerometers, compasses, and / or gyroscopes. In further techniques, the wireless device may use data derived from RF detection data to create or modify an indoor map. In some examples, the wireless device may download a partial indoor map of the indoor space and modify / edit the map to include references to any new objects / reflectors detected by the wireless device. In other embodiments, the wireless device may modify a partial indoor map to remove objects / reflectors that no longer exist in the indoor environment. In further embodiments, the wireless device may transmit the modified indoor map to a server that can aggregate the indoor map data with data received from other wireless devices.

[0112] In other embodiments, a wireless device that transmits and receives RF signals may transmit RF sensing data, device position data, and / or device orientation data to a server for further processing. In some examples, the server may use the RF sensing data and / or device orientation data received from the wireless device to calculate the distance and orientation of reflectors (e.g., objects) by calculating the time of flight and angle of arrival. In further examples, the server may create or modify an indoor map containing one or more references to newly identified objects and transmit the indoor map to the wireless device for presentation.

[0113] In some embodiments, the server may use artificial intelligence or machine learning algorithms to identify or classify reflective objects based on RF sensing data. The server may receive RF sensing data from a number of different wireless devices that can be provided as input to machine learning algorithms. In some examples, the server may perform "crowdsourcing" of data from a number of devices to develop a more detailed indoor map. Devices providing data to the server may or may not be associated with a local network (e.g., a Wi-Fi network) present in their respective indoor environments. For example, the server may receive data from devices located inside a building that provide local Wi-Fi, regardless of the device's association with a local network. The indoor map created and / or maintained by the server can be made accessible directly from the UE, accessible via the internet (e.g., a web page), accessible via a mobile or desktop application, and / or via any other suitable method that may be used to distribute the data.

[0114] In some examples, the processes described herein (e.g., processes 600, 700, and / or other processes described herein) may be performed by a computing device or apparatus (e.g., a UE). In one example, process 600 may be performed by the user device 207 in Figure 2. In another example, process 600 may be performed by a computing device with the computing system 800 shown in Figure 8. For example, a computing device with the computing architecture shown in Figure 8 may include components of the user device 207 in Figure 2 and be able to perform the operations shown in Figure 6.

[0115] In some cases, the computing device or apparatus may include a variety of components, such as one or more input devices, one or more output devices, one or more processors, one or more microprocessors, one or more microcomputers, one or more cameras, one or more sensors, and / or other components configured to perform the steps of the process described herein. In some examples, the computing device may include a display, one or more network interfaces configured to communicate and / or receive data, any combination thereof, and / or other components. One or more network interfaces may be configured to communicate and / or receive wired data and / or wireless data, including data compliant with 3G, 4G, 5G, and / or other cellular standards, data compliant with Wi-Fi (802.11x) standards, data compliant with Bluetooth® standards, data compliant with Internet Protocol (IP) standards, and / or other types of data.

[0116] The components of a computing device may be implemented in circuitry. For example, a component may include and / or be implemented using one or more programmable electronic circuits (e.g., a microprocessor, a graphics processing unit (GPU), a digital signal processor (DSP), a central processing unit (CPU), and / or other suitable electronic circuits), and / or may include and / or be implemented using computer software, firmware, or any combination thereof to perform the various operations described herein.

[0117] Process 600 is presented as a logical flow diagram, and its operation represents a set of operations that can be implemented by hardware, computer instructions, or a combination thereof. In the context of computer instructions, operation represents a computer-executable instruction stored on one or more computer-readable storage media that, when executed by one or more processors, performs the described operation. Generally, computer-executable instructions include routines, programs, objects, components, data structures, etc., that perform a particular function or implement a particular data type. The order in which operations are described is not intended to be interpreted as limiting, and any number of operations described may be combined in any order and / or in parallel to implement the process.

[0118] In addition, process 600, and / or other processes described herein, may be executed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is executed collectively on one or more processors, by hardware, or in combination thereof. As stated above, the code may be stored in a computer-readable or machine-readable storage medium, for example, in the form of a computer program comprising multiple instructions executable by one or more processors. The computer-readable or machine-readable storage medium may be non-temporary.

[0119] Figure 8 shows an example of a system for implementing several aspects of this technology. Specifically, Figure 8 shows an example of a computing system 800, which can be, for example, an internal computing system, a remote computing system, a camera, or any computing device that constitutes any of these components, and the components of the system communicate with each other using connections 805. Connections 805 can be a physical connection using a bus, or a direct connection to a processor 810 in a chipset architecture, for example. Connections 805 can also be a virtual connection, a network connection, or a logical connection.

[0120] In some embodiments, the computing system 800 is a distributed system in which the functions described herein can be distributed across a data center, multiple data centers, a peer network, and so on. In some embodiments, one or more of the described system components represent many components, each performing some or all of the functions that are the subject of the component description. In some embodiments, the components can be physical or virtual devices.

[0121] An exemplary system 800 includes at least one processing unit (CPU or processor) 810 and connection 805 that communicatively connects various system components, including system memory 815 such as read-only memory (ROM) 820 and random access memory (RAM) 825, to the processor 810. The computing system 800 may include a high-speed memory cache 812 that is directly connected to, near, or integrated as part of the processor 810.

[0122] The processor 810 may include any general-purpose processor and hardware or software services, such as services 832, 834, and 836, which are stored in memory device 830 and are configured to control the processor 810, as well as dedicated processors, such that software instructions are incorporated into the actual processor design. The processor 810 may essentially be a completely self-contained computing system including multiple cores or processors, buses, memory controllers, caches, etc. The multicore processor may be symmetric or asymmetric.

[0123] To enable user interaction, the computing system 800 includes an input device 845, which can represent any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, motion input, or speech. The computing system 800 may also include an output device 835, which can be one or more of a number of output mechanisms. In some cases, a multimodal system can enable a user to communicate with the computing system 800 by providing multiple types of input / output.

[0124] The computing system 800 may include a communication interface 840, which can generally control and manage user inputs and system outputs. The communication interface may include audio jacks / plugs, microphone jacks / plugs, Universal Serial Bus (USB) ports / plugs, Apple® Lightning® ports / plugs, Ethernet ports / plugs, fiber optic ports / plugs, proprietary wired ports / plugs, 3G, 4G, 5G, and / or other cellular data network wireless signal transmission, Bluetooth® wireless signal transmission, Bluetooth® Low Energy (BLE) wireless signal transmission, IBEACON® wireless signal transmission, Radio Frequency Identification (RFID) wireless signal transmission, Near Field Communication (NFC) wireless signal transmission, Dedicated Short Range Communication (DSRC) wireless signal transmission, 802.11 Wi-Fi wireless signal transmission, Wireless Local Area Network (WLAN) signal transmission, Visible Light Communication (VLC), and Worldwide Interoperability for Microwave. The system may perform or facilitate the reception and / or transmission of wired or wireless communications using wired and / or wireless transceivers, including those utilizing Access (WiMAX), infrared (IR) wireless signal transmission, public switched telephone network (PSTN) signal transmission, integrated services digital network (ISDN) signal transmission, ad hoc network signal transmission, radio wave signal transmission, microwave signal transmission, infrared signal transmission, visible light signal transmission, ultraviolet light signal transmission, wireless signal transmission along the electromagnetic spectrum, or any combination thereof.

[0125] The communication interface 840 may also include one or more GNSS receivers or transceivers used to determine the position of the computing system 800 based on the reception of one or more signals from one or more satellites associated with one or more Global Navigation Satellite Systems (GNSS). GNSS systems include, but are not limited to, the U.S. Global Positioning System (GPS), Russia's Global Navigation Satellite System (GLONASS), China's Beidou Navigation System (BDS), and Europe's Galileo GNSS. Since there are no restrictions on operation on any particular hardware configuration, this basic feature here may be easily superseded by improved hardware or firmware configurations as development progresses.

[0126] The storage device 830 can be a non-volatile and / or non-temporary and / or computer-readable memory device, such as a hard disk, magnetic cassette, flash memory card, solid memory device, digital multipurpose disk, cartridge, floppy disk, flexible disk, hard disk, magnetic tape, magnetic strip / stripe, any other magnetic storage medium, flash memory, memristor memory, any other solid-state memory, compact disc read-only memory (CD-ROM), optical disc, rewritable compact disc (CD) optical disc, digital video disc (DVD) optical disc, Blu-ray disc (BDD) optical disc, holographic optical disc, other optical media, Secure Digital (SD) card, microSecure Digital (microSD) card, Memory This can include other types of computer-readable media capable of storing computer-accessible data, such as Stick® cards, smart card chips, EMV chips, subscriber identification module (SIM) cards, mini / micro / nano / pico SIM cards, other integrated circuit (IC) chips / cards, random access memory (RAM), static RAM (SRAM), dynamic RAM (DRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash EPROM, cache memory (e.g., Level 1 (L1) cache, Level 2 (L2) cache, Level 3 (L3) cache, Level 4 (L4) cache, Level 5 (L5) cache, or other (L#) caches), resistive random access memory (RRAM / ReRAM), phase-change memory (PCM), spin-transfer torque RAM (STT-RAM), other memory chips or cartridges, and / or any combination thereof.

[0127] The storage device 830 may include software services, servers, services, etc., which cause the system to perform functions when code defining such software is executed by the processor 810. In some embodiments, a hardware service that performs a particular function may include software components stored on a computer-readable medium that are connected to the necessary hardware components, such as the processor 810, connection 805, and output device 835, in order to perform the function. The term “computer-readable medium” includes, but is not limited to, portable or non-portable storage devices, optical storage devices, and various other media that can store, contain, or carry instructions and / or data. Computer-readable medium may also include non-transient media that do not contain carrier waves and / or transient electronic signals that propagate wirelessly or via wired connections, and where data may be stored. Examples of non-transient media include, but are not limited to, magnetic disks or magnetic tapes, optical storage media such as compact discs (CDs) or digital multipurpose discs (DVDs), flash memory, memory, or memory devices. Computer-readable media may store code and / or machine-executable instructions, which may represent procedures, functions, subprograms, programs, routines, subroutines, modules, software packages, classes, or any combination of instructions, data structures, or program statements. Code segments may be coupled to other code segments or hardware circuits by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, transferred, or transmitted via any suitable means, including memory sharing, message passing, token passing, network transmission, etc.

[0128] Specific details have been provided in the above description to provide a complete understanding of the embodiments and examples provided herein, but it will be recognized by those skilled in the art that this application is not limited thereto. Therefore, while exemplary embodiments of this application have been described in detail herein, it should be understood that the concepts of the present invention may be embodied and employed in various other ways, and that, unless limited by the prior art, the appended claims are intended to be interpreted as including such variations. Various features and aspects of the applications described above may be used individually or together. Furthermore, embodiments may be used in any number of environments and applications other than those described herein without departing from the broader spirit and scope of this specification. Therefore, this specification and the drawings should be considered illustrative rather than restrictive. For illustrative purposes, the methods have been described in a particular order. It should be understood that in alternative embodiments, the methods may be performed in a different order than described.

[0129] For clarity of explanation, in some cases the technology may be presented as including individual functional blocks comprising a device, device components, and software, or steps or routines in a manner embodied in a combination of hardware and software. Additional components other than those shown in the figures and / or described herein may be used. For example, circuits, systems, networks, processes, or other components may be shown as components in the form of block diagrams, so as not to obscure the embodiment with unnecessary details. In other cases, well-known circuits, processes, algorithms, structures, and techniques may be shown without unnecessary details, so as not to obscure the embodiment.

[0130] Furthermore, those skilled in the art will understand that various exemplary logic blocks, modules, circuits, and algorithmic steps described in relation to the embodiments disclosed herein may be implemented as electronic hardware, computer software, or a combination of both. To clearly demonstrate this hardware and software compatibility, various exemplary components, blocks, modules, circuits, and steps are generally described above in relation to their functions. Whether such functions are implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functions in various ways for each specific application, but such decisions on implementation should not be construed as causing a departure from the scope of this disclosure.

[0131] Individual embodiments may be described above as processes or methods represented as flowcharts, flow diagrams, data flow diagrams, structural diagrams, or block diagrams. While flowcharts may describe operations as sequential processes, many operations can be performed in parallel or simultaneously. In addition, the order of operations may be rearranged. A process terminates when its operations are complete, but it may have additional steps not shown in the diagram. A process may correspond to a method, function, procedure, subroutine, subprogram, etc. When a process corresponds to a function, its termination may correspond to the function returning to a calling function or main function.

[0132] The processes and methods described above can be implemented using computer-executable instructions stored or otherwise available from a computer-readable medium. Such instructions may include instructions and data that cause a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or group of functions, or to otherwise configure a general-purpose computer, a dedicated computer, or a processing device to perform a particular function or group of functions. The portion of computer resources used may be accessible over a network. Computer-executable instructions may be binary or intermediate format instructions, such as assembly language, firmware, or source code. Examples of computer-readable media that may be used to store instructions, information used, and / or information created in the methods described above include magnetic or optical disks, flash memory, USB devices with non-volatile memory, and network-connected storage devices.

[0133] In some embodiments, computer-readable storage devices, media, and memory may include cables or wireless signals, such as bitstreams. However, non-transient computer-readable storage media, as referred to, inherently exclude media such as energy, carrier signals, electromagnetic waves, and signals.

[0134] Those skilled in the art will understand that information and signals may be represented using any of a variety of different techniques and methods. For example, data, instructions, commands, information, signals, bits, symbols, and chips, which may be mentioned throughout the above description, may, in some cases, be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or optical particles, or any combination thereof, depending in part to a specific application, in part to a desired design, in part to the corresponding technology, etc.

[0135] Various exemplary logic blocks, modules, and circuits described in relation to the embodiments disclosed herein may be implemented or executed using hardware, software, firmware, middleware, microcode, hardware description languages, or any combination thereof, and may take any of various form factors. When implemented in software, firmware, middleware, or microcode, program code or code segments (e.g., computer program products) for performing the required tasks may be stored in computer-readable or machine-readable media. A processor may perform the required tasks. Examples of form factors include laptops, smartphones, mobile phones, tablet devices or other small form factor personal computers, personal digital assistants, rack-mount devices, and standalone devices. The functions described herein can also be embodied in peripheral devices or add-in cards. Such functions can also, as a further example, be implemented on circuit boards in different chips or different processes running in a single device.

[0136] Instructions, a medium for transmitting such instructions, computing resources for executing such instructions, and other structures for supporting such computing resources are exemplary means for providing the functionality described herein.

[0137] The techniques described herein may also be implemented in electronic hardware, computer software, firmware, or any combination thereof. Such techniques may be implemented in any of a variety of devices, such as general-purpose computers, wireless communication device handsets, or integrated circuit devices having multiple applications, including applications in wireless communication device handsets and other devices. Any feature described as a module or component may be implemented together in an integrated logic device, or separately as individual but interoperable logic devices. When implemented in software, the technique may be at least partially implemented by a computer-readable data storage medium comprising program code that, when executed, includes instructions to perform one or more of the methods, algorithms, and / or operations described above. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise memory or data storage media such as random access memory (RAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, etc. The technique may, as an addition or alternative, be at least partially implemented by a computer-readable communication medium that carries or communicates program code in the form of instructions or data structures, such as propagating signals or waves, and which can be accessed, read, and / or executed by a computer.

[0138] The program code may be executed by a processor which may include one or more processors, such as one or more digital signal processors (DSPs), general-purpose microprocessors, application-specific integrated circuits (ASICs), field-programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuit configurations. Such processors may be configured to perform any of the techniques described herein. The general-purpose processor may be a microprocessor, but alternatively, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. Accordingly, the term “processor” as used herein may refer to any of the above structures, any combination thereof, or any other structure or device suitable for implementing the techniques described herein.

[0139] Those skilled in the art will understand that the symbols or terms less than ("<") and greater than (">") used herein may be replaced with the symbols less than or equal to ("≦") and greater than or equal to ("≧"), respectively, without departing from the scope of this description.

[0140] When a component is described as "configured to perform certain actions," such configuration can be achieved, for example, by designing electronic circuits or other hardware to perform the actions, by programming programmable electronic circuits (e.g., a microprocessor or other suitable electronic circuit) to perform the actions, or by any combination thereof.

[0141] The phrase "to be coupled" or "to be communicatively coupled to" refers to any component that is physically connected to another component, either directly or indirectly, and / or communicates, either directly or indirectly, with another component (for example, connected to another component via a wired or wireless connection and / or other appropriate communication interface).

[0142] The claim language or other wording that states “at least one of” the set and / or “one or more” of the set indicates that one element of the set or multiple elements of the set (in any combination) satisfy the claim. For example, the claim language that states “at least one of A and B” or “at least one of A or B” means A, B, or A and B. In another example, the claim language that states “at least one of A, B, and C” or “at least one of A, B, or C” means A, B, C, or A and B, or A and C, or B and C, or A and B and C. The phrase “at least one of” the set and / or “one or more” of the set does not limit the set to the items listed in the set. For example, the wording of a claim that states "at least one of A and B" or "at least one of A or B" could mean A, B, or A and B, and could further include items not listed in the set of A and B.

[0143] Exemplary embodiments of this disclosure include:

[0144] Embodiment 1: Apparatus for creating indoor maps. The apparatus includes at least one memory and at least one processor coupled to the at least one memory. The at least one processor is configured to receive a first set of radio frequency (RF) sensing data and orientation data corresponding to a first wireless device from a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and to generate an indoor map which includes a reference to the first reflector based on the first set of RF sensing data, orientation data and position data corresponding to the first wireless device.

[0145] Embodiment 2: Apparatus according to Embodiment 1, wherein the first set of RF detection data includes the distance and angle of arrival between the first wireless device and the first reflector.

[0146] Embodiment 3: The apparatus according to Embodiment 1, wherein at least one processor is configured to determine the distance and angle of arrival between a wireless device and a first reflector based on a first set of RF sensing data.

[0147] Embodiment 4: The apparatus according to any one of Embodiments 1 to 3, wherein at least one processor is configured to determine the movement of a first wireless device based on orientation data and position data, and to adjust the distance and angle of arrival based on the movement of the wireless device.

[0148] Embodiment 5: The apparatus according to any one of Embodiments 1 to 4, wherein at least one processor is configured to determine a second distance and a second angle of arrival between the second wireless device and the second reflector based on a second set of RF sensing data corresponding to the second wireless device, and to modify an indoor map to include a second reference of the second reflector in response to a determination that the first wireless device and the second wireless device are located in the same indoor environment.

[0149] Embodiment 6: The apparatus according to Embodiment 5, wherein the second wireless device is a Wi-Fi access point, and the first wireless device is not associated with the Wi-Fi access point.

[0150] Embodiment 7: The apparatus according to any one of Embodiments 1 to 6, wherein at least one processor is configured to receive a plurality of RF sensing datasets corresponding to a plurality of wireless devices, wherein the plurality of RF sensing datasets are associated with a plurality of received waveforms which are reflections of at least one transmitted waveform from a first reflector; to determine a plurality of position measurements associated with a first reflector based on the plurality of RF sensing datasets; and to determine the position of a first reflector based on the plurality of position measurements.

[0151] Embodiment 8: The apparatus according to Embodiment 7, wherein at least one processor is configured to determine that a first reflector corresponds to a transient object, and to modify an indoor map to remove a reference to the first reflector, based on a plurality of RF sensing datasets corresponding to a plurality of wireless devices.

[0152] Embodiment 9: The apparatus according to any one of Embodiments 7 and 8, wherein at least one processor is configured to determine, based on a plurality of RF sensing datasets corresponding to a plurality of wireless devices, that a first reflector corresponds to a structural element, and that the reference of the first reflector indicates the type of structural element.

[0153] Embodiment 10: The apparatus according to Embodiment 9, wherein the type of structural element includes at least one of doors, windows, walls, floors, ceilings, roofs, columns, or combinations thereof.

[0154] Embodiment 11: The apparatus according to any one of Embodiments 7 to 10, wherein at least one processor is configured to determine, based on a plurality of RF sensing datasets, that a first reflector corresponds to a static object, and that the reference of the first reflector indicates the type of static object.

[0155] Embodiment 12: The apparatus according to Embodiment 11, wherein the type of static object includes at least one of furniture, equipment, fixtures, or a combination thereof.

[0156] Embodiment 13: The apparatus according to any one of Embodiments 1 to 12, wherein the transmitted waveform includes at least one of a Wi-Fi signal, a New Radio (NR) signal, a Radar signal, an Ultra Wideband (UWB) signal, or any combination thereof.

[0157] Embodiment 14: The apparatus according to any one of Embodiments 1 to 13, wherein the first set of RF detection data includes channel status information (CSI) data.

[0158] Embodiment 15: The apparatus according to Embodiment 14, wherein the position data is determined based on CSI data.

[0159] Embodiment 16: The apparatus according to any one of Embodiments 1 to 15, wherein the location data includes relative positions inside a building.

[0160] Embodiment 17: The apparatus according to any one of Embodiments 1 to 16, wherein the orientation data is based on data obtained from a gyroscope on a wireless device, a compass on a wireless device, or any combination thereof.

[0161] Embodiment 18: The apparatus according to any one of Embodiments 1 to 17, wherein the transmitted waveform is transmitted by an access point (for example, a Wi-Fi access point).

[0162] Embodiment 19: The apparatus according to any one of Embodiments 1 to 17, wherein at least one processor is configured to provide an indoor map to a first wireless device.

[0163] Embodiment 20: The apparatus according to any one of Embodiments 1 to 19, wherein the first set of RF detection data includes data relating to at least one received leak waveform corresponding to a transmitted waveform.

[0164] Embodiment 21: A method for generating an indoor map, comprising the operation described in any one of Embodiments 1 to 20.

[0165] Embodiment 22: A computer-readable medium comprising at least one instruction for causing a computer or processor to perform an operation according to any one of Embodiments 1 to 20.

[0166] Embodiment 23: An apparatus for creating indoor maps, comprising means for performing operations according to any one of Embodiments 1 to 20.

[0167] Embodiment 24: Apparatus for creating indoor maps. The apparatus includes at least one memory, at least one transceiver, and at least one processor coupled to at least one memory and at least one transceiver. The at least one processor is configured to transmit radio frequency (RF) signals via at least one transceiver, receive a plurality of reflected RF signals via at least one transceiver, each of which is a reflection of a transmitted RF signal from at least one object in an indoor space, acquire RF detection data for the plurality of reflected RF signals from at least one object, and present an indoor map of the indoor space including references to at least one object, wherein the references to at least one object are based on the RF detection data.

[0168] Embodiment 25: The apparatus according to Embodiment 24, wherein at least one processor is configured to transmit RF detection data, device location data, and device orientation data to a server device.

[0169] Embodiment 26: The apparatus according to Embodiment 25, wherein at least one processor is configured to receive an indoor map of an indoor space, including references to at least one object, from a server device.

[0170] Embodiment 27: The apparatus according to any one of Embodiments 24 to 27, wherein at least one processor is configured to calculate the distance and angle of arrival between a wireless device and at least one object based on RF detection data, device position data, and device orientation data.

[0171] Embodiment 28: The device location data is based on RF detection data, as described in Embodiment 27.

[0172] Embodiment 29: The apparatus according to any one of Embodiments 24 to 29, wherein at least one processor is configured to download a partial indoor map of an indoor space from a server device and to modify the partial indoor map of an indoor space to include a reference to at least one object.

[0173] Embodiment 30: The apparatus according to Embodiment 29, wherein at least one processor is configured to transmit a modified indoor map, including a reference to at least one object, to a server device.

[0174] Embodiment 31: The apparatus according to any one of Embodiments 24 to 30, wherein the RF signal includes at least one of a Wi-Fi signal, a New Radio (NR) signal, a RADAR signal, an Ultra Wideband (UWB) signal, or any combination thereof.

[0175] Embodiment 32: The apparatus according to any one of Embodiments 24 to 31, wherein the RF detection data includes channel status information (CSI) data.

[0176] Embodiment 33: A method for generating an indoor map, comprising the operation described in any one of Embodiments 24 to 32.

[0177] Embodiment 34: A computer-readable medium comprising at least one instruction for causing a computer or processor to perform an operation according to any one of Embodiments 24 to 32.

[0178] Embodiment 35: An apparatus for creating indoor maps, comprising means for performing operations according to any one of Embodiments 24 to 32. [Explanation of Symbols]

[0179] 100 Communication Networks 102 Access Point (AP) 104a, 104b, 104c, 104d User Equipment (UE) 106 Communication Links 108 Wi-Fi Networks (WLAN) 110 Direct Wireless Link 112 Communication Link 122 Backhaul Link 160 base station 170 Core Network 172 servers 207 User Devices 270 Computing Systems 272 Input Devices 274 Subscriber Identification Module (SIM) 276 Modem 278 Wire Restaurant Seaba 280 Output Devices 282 Digital Signal Processor (DSP) 284 processors 286 memory devices 287 Antenna 288 signals, wireless signals 289 Bus 300 Wireless Devices 302 Object 304 Digital-to-Analog Converter (DAC) 306 RF Transmitter 308 Analog-to-Digital Converter (ADC) 310 RF Receiver 312 TX antenna 314 RX Antenna 316 TX waveform 318 RX waveform 320 TX leak signal 400 Indoor environment 402 Mobile Devices 404 Access Point (AP) 406 TX antenna 408 RX antenna 410 RF signal 412 Reflected signal 414 Leakage signal 415a, 415b, 415c signals 416a RF waveform 418a, 418b, 418c reflected waveform 420 Object 422 Wall 424 Wall 426 Wall 500 Graphical representations 600 processes 700 processes 800 Computing Systems 805 Connection 810 Processor, Processing Unit 812 cache 815 System Memory 820 Read-only memory (ROM) 825 Random Access Memory (RAM) 830 Storage Devices 832, 834, 836 Services 835 Output Device 840 Communication Interfaces 845 Input Devices

Claims

1. A device for creating indoor maps, At least one memory, The at least one memory is coupled to at least one processor, and the at least one processor is Receiving a first set of radio frequency (RF) sensing data and orientation data corresponding to a first wireless device of a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and the orientation data indicates the orientation of the first wireless device. The system is configured to generate an indoor map including a reference to the first reflector based on the first set of RF detection data, the orientation data, and the position data corresponding to the first wireless device, The aforementioned transmission waveform is transmitted by a Wi-Fi access point.

2. The apparatus according to claim 1, wherein the first set of RF detection data includes the distance and angle of arrival between the first wireless device and the first reflector.

3. The aforementioned at least one processor is The apparatus according to claim 1, configured to determine the distance and angle of arrival between the first wireless device and the first reflector based on the first set of RF detection data.

4. The aforementioned at least one processor is The movement of the first wireless device is determined based on the orientation data and the position data, The apparatus according to claim 3, configured to adjust the distance and the angle of arrival based on the movement of the first wireless device.

5. The aforementioned at least one processor is Based on a second set of RF detection data corresponding to the second wireless device, the second distance and second angle of arrival between the second wireless device and the second reflector are determined. The system is configured to modify the indoor map to include a second reference of the second reflector in response to a determination that the first wireless device and the second wireless device are located within the same indoor environment. The second wireless device is the Wi-Fi access point, and the first wireless device is not associated with the Wi-Fi access point. The apparatus according to claim 3.

6. The aforementioned at least one processor is Receiving a plurality of RF sensing datasets corresponding to the plurality of wireless devices, wherein the plurality of RF sensing datasets are associated with a plurality of received waveforms which are reflections of at least one transmitted waveform from the first reflector. Based on the plurality of RF detection datasets, a plurality of position measurements associated with the first reflector are determined, The apparatus according to claim 1, configured to determine the position of the first reflector based on the plurality of position measurement values.

7. The aforementioned at least one processor is Based on the plurality of RF detection datasets corresponding to the plurality of wireless devices, it is determined that the first reflector corresponds to a transient object. The apparatus according to claim 6, configured to modify the indoor map so as to remove the reference of the first reflector.

8. The aforementioned at least one processor is Based on the plurality of RF detection datasets corresponding to the plurality of wireless devices, the first reflector is determined to correspond to a structural element, wherein the reference of the first reflector is configured to indicate the type of the structural element. Optionally, the type of the structural element includes at least one of a door, window, wall, floor, ceiling, roof, column, or a combination thereof. The apparatus according to claim 6.

9. The aforementioned at least one processor is Based on the plurality of RF detection datasets, the system is configured to determine that the first reflector corresponds to a static object, wherein the reference of the first reflector indicates the type of the static object. Optionally, the type of the static object includes at least one of furniture, equipment, fixtures, or a combination thereof. The apparatus according to claim 6.

10. The apparatus according to claim 1, wherein the transmitted waveform further includes at least one of a New Radio (NR) signal, a RADAR signal, an Ultra Wideband (UWB) signal, or any combination thereof.

11. The aforementioned at least one processor is The apparatus according to claim 1, configured to provide the indoor map to the first wireless device.

12. A method for generating one or more indoor maps, The server device receives a first set of radio frequency (RF) sensing data and orientation data corresponding to a first wireless device of a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and the orientation data indicates the orientation of the first wireless device. The server device includes the step of generating an indoor map including a reference to the first reflector based on the first set of RF sensing data, the orientation data, and the position data corresponding to the first wireless device, The method by which the transmitted waveform is transmitted by a Wi-Fi access point.

13. The method according to claim 12, wherein the first set of RF detection data includes data relating to at least one received leak waveform corresponding to the transmitted waveform.

14. A computer-readable medium containing at least one instruction, wherein the at least one instruction is transmitted to a computer or processor. Receiving a first set of radio frequency (RF) sensing data and orientation data corresponding to a first wireless device of a plurality of wireless devices, wherein the first set of RF sensing data is associated with at least one received waveform which is a reflection of a transmitted waveform from a first reflector, and the orientation data indicates the orientation of the first wireless device. The system generates an indoor map including a reference to the first reflector based on the first set of RF detection data, the orientation data, and the position data corresponding to the first wireless device. The aforementioned transmission waveform is a computer-readable medium transmitted by a Wi-Fi access point.

Images