Novel systems and methods regarding location and operation of an intangible attribute source using extended reality

Abstract

Systems and methods for rendering a virtual three-dimensional space scene that includes location of one or more sources of intangible attributes are described. An exemplar method includes: (i) obtaining an intangible attribute dataset; (ii) modifying the intangible attribute dataset to produce a source identifying dataset; (iii) identifying, based on the source identifying dataset, one or more of different types of the intangible attribute sources that produce the intangible attributes present inside the real space; (iv) determining, based on the source identifying dataset, one or more locations and / or direction of locations of one or more of the intangible attribute sources; and (v) displaying and / or causing to display, in the virtual space defined inside a user interface of a user device, one or more of the locations and / or direction of locations of one or more of the intangible attribute sources and properties of one or more different types of the intangible attributes.

Description

NOVEL SYSTEMS AND METHODS REGARDING LOCATION AND OPERATION OF AN INTANGIBLEATTRIBUTE SOURCE USING EXTENDED REALITYRELATED APPLICATION

[0001] This application claims priority to provisional application number 63 / 619,321, filed on January 10, 2024, which is incorporated herein by reference for all purposes.FIELD

[0002] The present arrangements and teachings relate generally to novel systems and methods regarding the location and / or direction of location of an intangible attribute source (e.g., signal generator) using extended reality, which includes and spans from virtual reality to augmented reality. More particularly, the present arrangements and teachings relate to novel systems and methods, using among other techniques, complex data spatialization, for locating concealed, hidden, or unknown intangible attribute sources devices that generate a signal not visible to the human eye.BACKGROUND

[0003] Virtual reality (“VR”) and augmented reality (“AR”) conventionally use a human interface device to effectuate the full features of the technology. Further, each of VR and AR spans a range of reality, which is commonly called “Extended Reality” (“XR”). By way of example, user, using a display interface, will see the XR or virtual world, which broadly describes a world that encompasses both VR, AR, and all the intermediate realities.

[0004] Unfortunately, an XR system is typically limited and unable to process large amounts of complex data to convey information about the actual world (which is different from the virtual world) and therefore is not able to effectively inform about the actual world. As a result, applications of the XR system are limited and do not always lend themselves to conveying meaningful information. Moreover, given this limited bandwidth for data processing, it is difficult to design applications, which deploy XR systems, for commercially viable applications.

[0005] What is, therefore, needed are systems and methods that use XR systems and methods and that process large amounts of complex data to inform on real space, without suffering fromthe drawbacks encountered when using systems and methods involving the current XR technology.SUMMARY

[0006] The present systems and methods described herein relate to processing large amounts of complex data to inform, in a virtual space, on information present in, but not visible to a human eye, in a real space. Moreover, the present systems and methods deploy XR systems to display, in a two-dimensional or three-dimensional virtual space, locations and / or directions of locations of concealed, hidden, or unknown intangible attribute sources, e.g., Wi-Fi™ routers and / or tracking devices that generate a signal not visible to the human eye, using extended reality. Moreover, the present systems and methods deploy XR systems to also determine, in a virtual space, whether an intangible attribute source is properly operating.

[0007] In one aspect, the present teachings offer methods for determining a location and / or direction of location of one or more sources of intangible attributes. One such exemplar method includes: obtaining, for a real space, an intangible attribute dataset resulting from displacement, in the real space, of one or more vehicles. Each of the vehicles is equipped with or communicatively coupled to one or more non-imaging sensors that measure one or more properties of one or more different types of the intangible attributes present in the real space. These non-imaging sensor measurements are performed at different instances in time and at one or more three-dimensional coordinates that define a region or a location. Moreover, the intangible attribute dataset includes different instances in time of measurement, one or more the three-dimensional coordinates of measurement, one or more of the properties measured, and does not include imaging data of the real space.

[0008] Another element includes obtaining an image value dataset that conveys image- related information of the real space. The image value dataset includes one or more of the three- dimensional coordinates, which define the location or the region in the real space of the image- related information. Moreover, obtaining said image value dataset includes displacing one or more the vehicles and / or an imaging sensor that is equipped with or communicatively coupled to one or more the vehicles.

[0009] The exemplar method for rendering a three-dimensional space scene in the virtual space also includes a “modifying” element that modifies the intangible attribute dataset. In thiselement, a source identifying dataset is produced. The source identifying element includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement and information regarding one or more of the properties measured, and does not include imaging data of the real space.

[0010] The exemplar method for determining location and / or direction of location of one or more sources of intangible attributes then carries out identifying, based on the source identifying dataset and without using the imaging data, one or more of the different types of the intangible attribute sources that produce one or more different types of the intangible attributes present inside the real space.

[0011] Next, the exemplar method proceeds to determining, based on source identifying dataset and without using the imaging data, one or more locations and / or directions of locations of the one or more of the type of the intangible attribute sources and / or properties of one or more of the types of the intangible attributes.

[0012] Next, the exemplar method performs displaying and / or causing to display, in the virtual space defined inside a user interface, one or more the locations and / or the directions of the locations of one or more of the different types of the intangible attribute sources.

[0013] In one embodiment of the present teachings, one of the vehicles is an unmanned vehicle and the user device is a processor. In another embodiment of the present teachings, one of the vehicles is an aerial drone and the user device is a laptop. In yet another embodiment of the present teachings, the user device is an augmented and virtual reality headset and the virtual space is a three-dimensional spatialized environment.

[0014] In the element of obtaining, the intangible attribute is at least one attribute chosen from a group including radio signal, throughput of a connectivity signal, latency of the connectivity signal, interference of the connectivity signal, volatility of the connectivity signal, stability of the connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature.

[0015] One or more of the properties of one or more different types of intangible attributes, in one embodiment of the present teachings is at least one property chosen from a groupincluding frequency regime, wavelength, amplitude, signal strength, phase angle, concentration, pressure, and phase difference.

[0016] In one exemplar method, the intangible attribute is a signal propagating in the real space. The element of displaying and / or causing to display may include displaying two or more properties of the signal propagating in the real space. The property of the signal propagating in the real space includes at least one property chosen from a group including, frequency regime of the signal propagating in the real space, strength of the signal propagating in the real space, wavelength of the signal propagating in the real space, phase angle of the signal propagating in the real space, phase difference of the signal propagating in the real space, angle of arrival of the signal propagating in the real space, peak frequency of the signal propagating in the real space, and direction of the signal propagating in the real space, and name of the source of the signal propagating in the space.

[0017] Determining one or more locations and / or directions of locations of one or more of the different types of the intangible attribute sources, in one implementation of the present teachings, includes interpolating, using two or more portions of the intangible attribute dataset and / or the source identifying dataset associated with a particular type of the intangible attribute source and with two or more corresponding three-dimensional location coordinates, to compute a predicted intangible attribute measurement associated with the particular type of the intangible attribute source at a corresponding interior three-dimensional location coordinate that is disposed inside a confined region within the real space. The predicted intangible attribute measurement is positioned on a linear trajectory that intersects two or more of the corresponding three- dimensional location coordinates. In this implementation, the real space is defined using two or more of the corresponding three-dimensional location coordinates. The corresponding interior three-dimensional location coordinate associated with the particular type of the intangible attribute source, in one embodiment, is relative to one or more three-dimensional coordinates that define a perimeter or boundary of the confined region.

[0018] The element of displaying and / or causing to display presents an interior virtual object at a corresponding interior three-dimensional location coordinate associated with the particular type of the intangible attribute source and the interior virtual objects includes arrows and / or an illustration of the particular type of the intangible attribute source.

[0019] Determining one or more locations and / or directions of locations of one or more of the different types of the intangible attribute sources, in another implementation of the present teachings, includes extrapolating, using two or more portions of the intangible attribute dataset and / or the source identifying dataset associated with a particular type of the intangible attribute source and with two or more corresponding three-dimensional location coordinates, to compute a predicted intangible attribute measurement associated with the particular type of the intangible attribute source at a corresponding exterior three-dimensional location coordinate that is disposed outside the real space. The predicted intangible attribute measurement is positioned on a linear trajectory that intersects two or more of the corresponding three-dimensional location coordinates. Additionally, the real space is defined using two or more of the corresponding three- dimensional location coordinates.

[0020] The element of displaying and / or the causing to display presents an exterior virtual object at a corresponding exterior three-dimensional location coordinate associated with the particular type of the intangible attribute source and the exterior virtual objects includes arrows and / or an illustration of the particular type of the intangible attribute source.

[0021] In one preferred embodiment of the present teachings, the modifying element further includes performing a mathematical transformation on the intangible attribute dataset set to produce the source identifying dataset, wherein the performing the mathematical transformation using at least one transformation chosen from a group including fast Fourier transform, discrete Fourier transform, and discrete cosine transform.

[0022] The modifying element in this exemplar method includes: 1) filtering the intangible attribute dataset to produce a filtered intangible attribute dataset wherein the filtering removes, within the intangible attribute dataset, data that is spatially and / or temporally substantially similar, wherein the filtering is carried out after the performing the mathematical transformation; and (2) modifying the filtered intangible attribute dataset to produce the source identifying dataset.

[0023] Further, the identifying element may include matching at least a part of the source identifying dataset with one or more reference source identifying datasets, wherein one or more of the reference sources identifying datasets is generated by a particular type of intangible attribute source.

[0024] Matching at least a part of the source identifying dataset with one or more reference source identifying datasets, one implementation of the present teachings, includes matching, within a predefined tolerance, at least the part of the source identifying dataset with a spatial reference pattern generated by the particular type of the intangible attribute source and / or a temporal reference pattern generated by the particular type of intangible attribute source. The spatial reference pattern may include one or more reference spatial peaks, and each of the reference spatial peaks has a reference spatial value. The temporal reference pattern includes one or more reference temporal peaks and each of the reference temporal peaks has a reference temporal value. Matching may include matching, within the predefined tolerance, at least the part of the source identifying dataset with one or more of the reference spatial values associated with the spatial reference pattern and / or with one or more of the reference temporal values associated with the temporal reference pattern, the source identifying dataset may include one or more spatial patterns, each of which includes one or more spatial peaks having a spatial value and a spatial strength value (e.g., as signal strength or amplitude value) and / or one or more temporal patterns, each of which includes one or more temporal peaks having a temporal value and a temporal strength value.

[0025] In a preferred embodiment, the element of displaying and / or the causing to display element includes: (1) spatializing the identifying dataset to create a spatialized intangible attribute dataset; (2) obtaining an image value dataset that conveys image-related information of the real space and includes one or more of the three-dimensional coordinates, which define a location or a region in the real space of the image-related information; (3) spatializing, using plurality of the three-dimensional coordinates, the image value dataset to create a spatialized image value dataset; (4) aligning the spatialized image value dataset with the spatialized intangible attribute dataset to create a revealing three-dimensional spatialized environment that reveals one or more locations or approximate locations of one or more of the intangible attribute sources; and (5) rendering and / or causing to rendering, using a rendering engine and in the virtual space inside the user interface, the three-dimensional spatialized environment identifying one or more the locations and / or the directions of the locations of one or more of the different types of the intangible attribute sources.

[0026] In another aspect, the present teachings offer methods determining the location of one or more unknown audio and / or visual recording devices (“unknown devices”) in a real space.One such exemplar method includes: (i) displacing one or more vehicles within a three- dimensional real space, each or which is equipped with or communicatively coupled to one or more non-imaging sensors that measure, at one or more said three-dimensional coordinates that define a region or a location, one or more properties of one or more different types of said intangible attributes present in said real space; (ii) obtaining an unknown device dataset resulting from said displacing of one or more of said vehicles in said three-dimensional real space, wherein said unknown device dataset includes different instances in time of measurement, one or more of said three-dimensional coordinates of measurement, one or more said properties measured, and does not include imaging data of said real space; (iii) modifying the unknown device dataset to produce a source identifying dataset, which includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement and, one or more temporal frequency patterns and / or one or more spatial frequency patterns associated with one or more of the properties of one or more of the different types of intangible attributes present in the real space, and does not include imaging data of the real space; (iv) identifying, based on one or more of the spatial frequency patterns and / or one or more of the temporal frequency patterns and without using the imaging data, one or more of the unknown devices that transmit the intangible attribute inside the real space; (v) determining, using said source identifying dataset and without using said imaging data, one or more locations and / or directions of locations of said one or more of said unknown devices and / or frequency and / or strength of one or more different types of said intangible attributes of one or more of said unknown devices; (vi) obtaining an image value dataset that conveys image-related information of the real space and includes one or more of the three-dimensional coordinates, which define a location or a region in the real space of the image-related information; (vii) spatializing, using plurality of the three-dimensional coordinates and a spatialized model, the image value dataset to create a spatialized image value dataset; (viii) spatializing, and the source identifying dataset and the spatialized model, to create a spatialized source identifying dataset; (ix) aligning the spatialized image value dataset with the spatialized source identifying dataset to create a revealing three-dimensional spatialized environment that reveals one or more locations or approximate locations of one or more of the intangible attribute sources; and (x) displaying and / or causing to display, using a rendering engine and in a virtual space inside a user interfaceof a user device that is communicatively coupled to one or more of the vehicles, the three- dimensional spatialized environment identifying one or more the locations and / or the directions of the locations of unknown devices and properties of one or more of the types of said intangible attributes.

[0027] In the exemplar embodiment described above, the identifying includes matching, within a predefined tolerance, one or more of the spatial frequency patterns and / or one or more of the temporal frequency patterns with a spatial frequency reference pattern generated by the particular type of unknown device and / or a temporal frequency reference pattern generated by the particular type of the unknown device.

[0028] In yet another aspect, the present arrangements offer systems for determining a location of one or more sources of intangible attributes. One such exemplar system includes: (i) one or more vehicles, each of which is equipped with or communicatively coupled to one or more non-imaging sensors for measuring, at different instances in time and at one or more three- dimensional coordinates that define a region or a location in a real space, one or more properties of one or more different types of said intangible attributes present in the real space to produce an intangible attribute dataset and does not include imaging data of the real space; (ii) an imaging system for obtaining an image value dataset that includes the imaging data of the real space and conveys image-related information of the real space and includes one or more of the three- dimensional coordinates, which define a location or a region in the real space of the image- related information; and (iii) a processor communicatively coupled to the non-imaging sensor and the imaging system, and programmed for carrying out following instructions: (a) obtaining, within the real space, the intangible attribute, wherein the intangible attribute dataset includes different instances in time of measurement, one or more the three-dimensional coordinates of measurement, one or more of the properties measured, and does not include imaging data of the real space; (b) modifying the intangible attribute dataset to produce a source identifying dataset, which includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement and information regarding one or more of the properties measured, and does not include imaging data of the real space; (c) identifying, based on the source identifying dataset and without using the imaging data, one or more of the different types of the intangible attribute sources that produce one or more different types of the intangibleattributes present inside the real space; (d) determining, based on source identifying dataset and without using the imaging data, one or more locations and / or directions of locations of the one or more of the type of the intangible attribute sources and / or properties of one or more of the types of the intangible attributes; and (e) displaying and / or causing to display, in the virtual space defined inside a user interface, one or more the locations and / or the directions of the locations of one or more of the different types of the intangible attribute sources and properties of one or more of the types of the intangible attributes.

[0029] In one embodiment, the element of displaying and / or the causing to display is carried out using a display interface.

[0030] Preferably, the imaging system includes one or more imaging sensors mounted on one or more of the vehicles.

[0031] The construction and method of operation of the invention, however, together with additional objects and advantages thereof, will be best understood from the following descriptions of specific embodiments when read in connection with the accompanying figures.BRIEF DESCRIPTION

[0032] Figure 1 shows a block diagram of an XR system, according to one embodiment of the present arrangements and that includes, among other things, a mobile device and a processor to create XR.

[0033] Figure 2 shows a mobile device, according to one embodiment of the present arrangements and that includes a vehicle and imaging devices.

[0034] Figure 3 shows a block diagram of a processor-based mobile device, according to one embodiment of the present arrangements that integrates a non-imaging sensor, a processor, and an optical sensor, which includes the imaging devices of Figure 2, such that these components function in a cooperative manner to perform certain methods of the present teachings described herein.

[0035] Figure 4A shows a block diagram of the various software modules and a rendering engine, according to one embodiment of the present arrangements, present inside a processor, e.g., inside the processor-based mobile device of Figure 3 or the server of Figure 1.

[0036] Figure 4B shows a block diagram of the various software modules and a rendering engine, according to an alternative embodiment of the present arrangements, present inside a processor, e.g., inside the processor-based mobile device of Figure 3 or the server of Figure 1.

[0037] Figure 5 shows an XR system, according to one embodiment of the present arrangements, that has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0038] Figure 6 shows an XR system, according to another embodiment of the present arrangements and that includes a mobile device communicatively coupled to a smartphone, which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0039] Figure 7 shows an XR system, according to yet another embodiment of the present arrangements and that includes a mobile device communicatively coupled to a network e.g., the Internet), which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0040] Figure 8 shows an XR system, according to yet another embodiment of the present arrangements and that includes a mobile device communicatively coupled to a personal computer, which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0041] Figure 9 shows an XR system, according to yet another embodiment of the present arrangements, and that has stored therein programmable instructions for carrying out, in the absence of a mobile device, certain methods of the present teachings described herein.

[0042] Figure 10 shows an exemplary visual representation resulting from a cluster analysis, according to one embodiment of the present teachings and that shows different types of intangible attribute sources ( .g., Wi-Fi™ signal generator and / or Bluetooth™ signal generator) present in real space.

[0043] Figure 11 shows a two-dimensionally rendered visual representation of a virtual space, according to one embodiment of the present teachings, displayed on a display of an XR system identifying locations and directions of different types of intangible attribute sources, which are sources of different types of properties (e.g., period of the signal and / or amplitude of the signal) of an intangible attribute (e.g., radio waves, such as Wi-Fi™ signals, providingwireless high-speed Internet access and / or radio waves, such as Bluetooth™ signals, enabling short-range wireless communication between electronic devices) present in real space.

[0044] Figure 12 shows a two-dimensionally rendered visual representation of a virtual space, according to another embodiment of the present teachings, displayed on a display of an XR system identifying locations and directions of different types of intangible attribute sources within a structure, which are sources of different types of properties (e.g., period of the signal and / or amplitude of the signal) of an intangible attribute e.g., radio waves, such as Wi-Fi™ signals, providing wireless high-speed Internet access and / or radio waves, such as Bluetooth™ signals, enabling short-range wireless communication between electronic devices) present within the structure in real space.

[0045] Figure 13 shows a rendered three-dimensional spatialized image, according to one embodiment of the present arrangements and that depicts a distribution of WiFi-connectivity signal strength indicators surrounding a particular real space of interest.

[0046] Figure 14 shows a process flow diagram for a method, according to one embodiment of the present teachings, for displaying and / or causing to display, in a virtual space, defined inside an XR system (e.g., a display) one or more locations and / or directions of locations of one or more intangible attribute sources and one or more properties of one or more intangible attribute sources.

[0047] Figure 15 shows an intangible attribute dataset, according to one embodiment of the present teachings, that includes multiple intangible attribute measurements at different coordinates in real space.

[0048] Figure 16 shows a spatially distributed reference table, according to one embodiment of the present teachings, for a characteristic frequency regime (e.g., Fl) at a particular time, (e.g., t=Tl).

[0049] Figure 17 shows a temporally distributed reference table, according to one embodiment of the present teachings, for the characteristic frequency regime of Figure 14 at a position in real space to develop a temporally distributed reference frequency pattern for this frequency regime.

[0050] Figure 18 shows an exemplary graph illustrating a radio signal in the time domain, according to one embodiment of the present teachings, from one or more intangible attribute sources.

[0051] Figure 19 shows an exemplary graph illustrating the radio signal of Figure 15 in the frequency domain, according to one embodiment of the present teachings.

[0052] Figure 20 shows a process flow diagram for a method, according to one embodiment of the present teachings, for determining a location of one or more sources of intangible attributes and / or one or more intangible attributes properties using an XR system.

[0053] Figure 21 shows a process flow diagram for a method, according to one embodiment of the present teachings, for spatializing and then aligning a source identifying dataset with an image value dataset to create a revealing spatialized environment that is rendered inside a mobile device of an XR system.DETAILED DESCRIPTION

[0054] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without limitation to some or all of these specific details. In other instances, well known process elements have not been described in detail in order to not unnecessarily obscure the invention.

[0055] The present arrangements provide systems of many different configurations for rendering extended realities (“XRs”) of the present teachings. In one XR of the present teachings, two or more different types of datasets are rendered. In one such XR, one dataset may include image data of a real space and another dataset may include data conveying a magnitude of an intangible property inside the real space. In another XR of the present teachings, an image dataset, which is an electronic representation of the real space being perceived by the user, is spatialized and rendered. According to the present teachings, image datasets are not the only type of datasets that may be spatialized before rendering. In fact, one or more different types of intangible attribute datasets present in the real space may be spatialized before rendering.

[0056] The present systems and methods described herein may be deployed using processorbased mobile devices (e.g., a vehicle). In one embodiment of the present arrangement, however, a smartphone, in the absence of a mobile device, renders one or more different types of datasets. As a result, use of the mobile device is not necessary, but represents a preferred embodiment of the present arrangements.

[0057] In certain embodiments, the processor-based mobile device of the present arrangements will generally comprise a processor, which includes one or more memory devices operable to provide machine-readable instructions to the processors and to store data. In some of these embodiments, the processor-based mobile device is communicatively coupled to external devices that include one or more sensors for collecting persisting data. In other embodiments, all the necessary sensors are integrated into the processor-based mobile device.

[0058] In one preferred embodiment of the present arrangements, the processor-based mobile device may include data acquired from remote servers. The processor may also be coupled to various input / output (“I / O”) devices for receiving input from a user or another system and for providing an output to a user or another system. These I / O devices may include human interaction devices such as keyboards, touch screens, displays, and terminals as well as remote connected computer systems, modems, radio transmitters, and handheld personal communication devices such as cellular phones, smartphones, and digital assistants.

[0059] The processors of the present arrangements may also include mass storage devices that are associated with disk drives and flash memory modules as well as connections through I / O devices to servers or remote processors containing additional storage devices and peripherals.

[0060] Certain embodiments may employ multiple servers and data storage devices thus allowing for operation in a cloud or for operations drawing from multiple data sources. The present teachings and arrangements contemplate that the methods disclosed herein will also operate over a network such as the Internet, and may be effectuated using combinations of several processing devices, memories, and I / O. Moreover, any device or system that operates to effectuate one or more elements, according to the present teachings, may be considered a “server,” as this term is used in this specification, if the device or system operates to communicate all or a portion of the programmable instructions to another device, such as a mobile device, eyewear, or a smartphone.

[0061] In certain aspects of the embodiments that employ multiple devices, i.e., multiple mobile devices, servers, and data storage devices may operate in a cloud or may operate, in the absence of the Internet, to draw from multiple data sources. In these configurations, multiple devices may collectively operate as part of a peer-to-peer ad-hoc network. These configurations of the present arrangement would implement “edge computing,” which is a distributedcomputing paradigm that brings computation and data storage closer to the location where it is needed, to improve response times and save bandwidth. In some of these aspects of the present arrangement, the mobile device would communicate with each other without the Internet, where one of them would act as a sort of communication server, but without any other connectivity.

[0062] The processors of the present system may be, partially or entirely, integrated into a wireless device such as a smartphone, a mobile device, a personal digital assistant (PDA), a laptop, a notebook, or a tablet computing device that operates through a wireless network. Alternatively, the entire processing system may be self-contained on a single device in certain embodiments of the present arrangements.

[0063] The processors of the present systems may be coupled to a three-dimensional (“3D”) controller for interactivity. A 3D controller takes human input and provides signals directing the processing system to alter a display of information. Conventional 3D controllers may be used to virtually move through an image displayed on a screen. Described below are certain exemplary embodiments of the processor-based mobile device of the present arrangements.

[0064] Figure 1 shows a functional block diagram of an Extended Reality (“XR”) system 100, according to one embodiment of the present arrangements. XR system 100 includes a server 110 that is coupled to one or more databases 112 and a network 114 (e.g., the Internet). Network 114 may include routers, hubs, and other equipment to effectuate communications between all associated devices. A user may choose to access server 110 by a computer 116 that is communicatively coupled to network 114. Computer 116 includes, among other things, a sound capture device such as a microphone (not shown to simplify illustration). Alternatively, the user may access server 110 through network 114 by using a smartphone 118. Smartphone 118 may connect to server 110 through an access point 120 coupled to network 114. Smartphone 118, like computer 116, may include a sound capture device such as a microphone.

[0065] XR system 100 may include one or more mobile devices (e.g., an aerial drone) 122 coupled to network 114 directly, through access point 120, or directly to remote processing devices. By way of example, a mobile device 122 or game controller may be coupled to a processing device for getting user input. This coupling may be wireless using technologies such as “Wi-Fi™,” “Bluetooth™,” or Z-Wave, i.e., a wireless technology standard used for exchanging data between fixed and mobile devices over short distances, using short- wavelengthUHF radio waves in the industrial, scientific, and medical radio bands, from 2.402 Gigahertz to 2.480 Gigahertz, and building personal area networks (PANs).

[0066] Computer 116, smartphone 118, and mobile device 122 may be referred to as client devices within XR system 100. Conventionally, client device / server processing operates by dividing the processing between two devices such as a server and a smart device, such as a smartphone or other computing device. The workload is divided between the servers and the client devices (e.g., mobile device, smartphones or wearables, such as eyewear), according to a predetermined specification. By way of example, in a “light client” mode of operation, the server does most of the data processing and the client device does a minimal amount of processing, often merely displaying and / or rendering the result of processing performed on a server (e.g., Server 110).

[0067] According to the current disclosure, client device / server applications are structured so that the server provides programmable instructions to one or more different types of client devices and the client devices execute those programmable instructions. The interaction between the server and the client device indicates which instructions are transmitted and executed. In addition, the client device may, at times, provide programmable instructions to the server, which in turn executes them. Several forms of programmable instructions are conventionally known including applets and are written in a variety of languages including Java and JavaScript.

[0068] Client device applications implemented on the client device, and / or server applications implemented on the server, provide for software as a service (SaaS) application where the server provides software to the client device on an as needed basis.

[0069] In addition to the transmission of instructions, client device / server applications also include transmission of data between the client device and server. Often this entails data stored on the client device to be transmitted to the server for processing. The resulting data is then transmitted back to the client device for display or further processing.

[0070] The present teachings recognize that client devices (e.g., computer 116, smartphone 118, or mobile device 122) may be communicably coupled to a variety of other devices and systems such that the client receives data directly and operates on that data before transmitting it to other devices or servers. Thus, data to the client device may come from input data from a user, from a memory on the device, from an external memory device coupled to the device, from a radio receiver coupled to the device, or from a transducer or sensor coupled to the device. In thepresent arrangements, the sensors may be optical sensors and / or non-imaging sensors. The radio may be part of a wireless communications system such as a “Wi-Fi™” or Bluetooth™ receiver. “Wi-Fi™,” as this term is used in this specification, refers to a family of wireless networking technologies, based on the IEEE 802.11 family of standards, which are commonly used for local area networking of devices and Internet access.

[0071] Sensors that are communicatively coupled to any one of the mobile devices, another client device (e.g., a smartphone or a personal computer), or a server may be any of a number of devices or instruments, in one embodiment, non-imaging sensors of the present arrangements detect a magnitude of an intangible property in the real space.

[0072] A client-server system, in which a client device and a server are communicatively coupled to achieve a desired result, may rely on “engines” that include processor-readable instructions (or code) to effectuate different elements of a design. Each engine may be responsible for different operations and may reside, in whole or in part, on a client device, a server, or another device. As disclosed herein a rendering engine, a display engine, a data engine, an execution engine, and a user interface (UI) engine may be employed. These engines may seek and gather information about events from remote data sources. Further, these engines may facilitate the rendering, displaying, and / or causing to display of “processed information,” which is generated by a processor.

[0073] The described methods and systems, of the present teachings, may be effectuated using conventional programming tools including database tools for collecting, storing, and searching through structured data. Moreover, web-based or other programming techniques may be employed to collect information and cause to display, display, or render results. Accordingly, software engines may be created to effectuate these methods and techniques, either in whole or part, depending on the desired embodiment described herein.

[0074] Figure 2 shows an exemplary mobile device 222 of the present arrangements that is substantially similar to mobile device 122 of Figure 1. Mobile device 222, as shown in Figure 2, includes an unmanned aerial vehicle (“UAV’) 202 communicatively that coupled to a payload 204. One or more non-imaging sensors 206 and one or more imaging sensors 208 are disposed within payload 204.

[0075] In one embedment of the present arrangements, mobile device 222 includes a one or more propulsion systems (e.g., motor that engages a propeller) that, when operating, displacemobile device 222 within a real space. Mobile device 222 may include, without limitation, a manned ground vehicle (e.g., a truck), an unmanned ground vehicle, an UAV, an unmanned underwater vehicle, airplane, helicopter, and sailboat. In another embodiment of the present arrangements, one or more of the propulsion systems moves mobile device 222 to a position or location where different forces move mobile device 222 within the real space. By way of example, mobile device 222 may be, without limitation, a high-altitude balloon or an earth orbit satellite. In yet another embodiment, mobile device 222 is coupled to an object that generates movement. By way of example, mobile device is mounted to a person or animal (e.g., a dog) that is moving through the real space.

[0076] Each of one or more non-imaging sensors 206 and one or more imaging sensors 208 may be positioned at various locations on mobile device 222 and / or payload 204. A line of sight of each imaging sensor 208 and / or non-imaging sensor 206 may depend on the type of mobile device 222 being used. For example, when mobile device 222 is a UAV, each of one or more non-imaging sensors 206 and one or more imaging sensors 208 may be located on the bottom of mobile device 222 and / or payload 204 and have a line of sight in a downward direction.

[0077] By way of another example, mobile device 222, in one exemplar embodiment of the present arrangements, includes six imaging sensors 208 (i.e., cameras) to capture an image of the space surrounding mobile device 222. A left look-side imaging sensor provides an image of the space that lies to the left side of mobile device 222, left look-front imaging sensor 208 provides an image of the space that lies in front, but to the left, of mobile device 222, and left look-down imaging sensor 208 provides an image of the space that lies below, but to the left, of mobile device 222. Similarly, with respect to the right side of mobile device 222, right look-side imaging sensor 208 provides an image of the space that lies to the right side of mobile device 222, right look-front imaging sensor 208 provides an image of the space that lies in front, but to the right, of mobile device 222, and right look-down imaging sensor 208 provides an image of the space that lies below, but to the right, of mobile device 222. Non-imaging sensors 206 may be oriented in similar directions.

[0078] In preferred configurations of these embodiments, mobile device 222 and / or payload 204 houses a processor, which serves as a processing subsystem for processing both imaging and non-imaging information obtained using above-described optical sensor (including an imaging device) and non-imaging sensor, respectively. In this embodiment, the processor of mobiledevice 222 not only generates “processed information” from the imaging and non-imaging information, but also effectively renders the “processed information” for displaying on a display.

[0079] Figure 3 shows a processor-based mobile device 322, according to one embodiment of the present arrangement and that includes a processing subsystem 313 that is configured to receive and process both imaging information obtained from an optical sensor device 305 and non-imaging device information or, in the alternative, non-imaging information, obtained from a non-imaging sensor device 340 (e.g. non-imaging sensor 208 of Figure 2) . Such processing of information allows processing subsystem 313 to produce the “processed information” that is rendered and / or displayed, or caused to display, on an I / O interface 317 (e.g., on a display screen or AR / VR headset). A network (e.g., the Internet), preferably, serves as a communication bus 365 to effectively communicatively couple processing subsystem 313, optical sensor device 305, non-imaging sensor device 340, and I / O interface 317 so that the requisite information from the sensors is received, processed and then displayed and / or rendered on an I / O interface 317.

[0080] Non-imaging sensor device 340 is a sensor that is configured to measure a magnitude of one or more intangible properties present within a real space. Non-imaging sensor device 340 may be hardware, software, or a combination of software and hardware. By way of example, non-imaging sensor devices 340 may represent, or include, radio frequency sensors (“RF sensors”) configured to receive and detect the radio signals (or RF signals). Examples of RF sensors may include, but are not limited to, RF antennas and RF identity (RFID) readers. The RF sensors may function as transducers to convert electromagnetic (EM) signals into electrical signals for processing by the processor. However, in one implementation of the present arrangements, at least one of the non-imaging sensor devices 340 may be implemented as a sensor device including a processor. For example, the non-imaging sensor device (e.g., the nonimaging sensor device 340 of Figure 3) may include RF sensors combined with a processor in a single package to receive, detect, and measure EM signals in the radio frequency range.

[0081] In preferred embodiments of the present arrangements, non-imaging sensor device 340 is at least one sensor chosen from a group including sensor measuring throughput of a connectivity signal, sensor measuring latency of a connectivity signal, sensor measuring interference of a connectivity signal, sensor measuring volatility of a connectivity signal, sensor measuring stability of a connectivity signal, radio frequency (hereinafter also referred to as “RF”) power output sensor, a radio frequency receiver sensor (e.g., a RF receiver module or a RFtransceiver module), electric and magnetic fields (“EMF”) sensor, atmospheric pressure sensor, geomagnetic sensor, hall effect sensor, ambient light level sensor, gas levels sensor, smoke sensor, sound pressure sensor, audio harmonics sensor, humidity sensor, carbon dioxide emission sensor, and temperature sensor. In one embodiment of the present arrangements, a sensor measuring a throughput of a connectivity signal measures a bandwidth of the connectivity signal, and sensor measuring volatility of a connectivity signal measures the stability of the connectivity signal.

[0082] In another embodiment of the present teachings, one or more non-imaging sensor devices 340 are distinct and / or located remote from mobile device 322. For example, a first nonimaging sensor device 340 may be mounted toward a left side of mobile device 322, such as the left side of a truck, and a second non-imaging sensor device 340 may be mounted toward a right side of mobile device 322, such as the left side of a truck. In yet another embodiment of the present teachings, a first non-imaging sensor device 340 is mounted onto mobile device 322 and a second non-imaging sensor device 340 is located on a remote device / surface or another mobile device 322. The first non-imaging sensor device 340 may be placed at a preset distance (or baseline distance) from the second non-imaging sensor device 340 sensor. The baseline distance may be stored in computer memory for access and / or retrieval by the processor (not shown) operating in communication with the non-imaging sensors. The processor may be located either on mobile device 322 (e.g., in the processing subsystem 313 of Figure 3) or remote therefrom. As will be discussed in greater detail below, the processor may be configured to determine a location and a direction of that location of one or more intangible attribute sources.

[0083] Optical sensor device 305 of Figure 3 includes an imaging sensor 315 e.g., imaging sensor 208 of Figure 2), an inertial measurement unit (“IMU”) 342, and one or more position sensors 335. Processing subsystem 313 includes an image processing engine 319, an application store 321, and a tracking module 323. Optical sensor device 305 may optionally include an optical assembly 330 and / or an electronic display 325.

[0084] In optical sensor device 305, one or more imaging sensors 315 capture data characterizing a scene or a local area of interest, typically the space surrounding a user. In accordance with one present arrangement, imaging sensor 315 includes a traditional image sensor e.g., camera), such that the signals captured by imaging sensor 315 include only two- dimensional image data (e.g., data having no depth information). In another present arrangement,imaging sensor 315 includes a projector device that allows imaging sensor 315 to operate as a depth imaging system that computes depth information for a scene using collected data (e.g., based on captured light according to one or more computer-vision schemes or algorithms, by processing a portion of a structured light pattern, by time-of-flight (“ToF”) imaging, or by Simultaneous Localization And Mapping (“SLAM”)). In an alternative present arrangement, imaging sensor 315 transmits corresponding data to another device, such as the processing subsystem 313, which determines or generates the depth information using the data from the imaging sensor 315.

[0085] Imaging sensor 315, in yet another present arrangement, is a light detection and ranging (“LiDAR”) sensor that transmits a pulse of infrared light and receives data that is refracted back to the LiDAR sensor to create a three-dimensional map of a scene or local area of interest. In yet another embodiment of the present arrangements, imaging sensor 315 is a radio detection and ranging (“RADAR”) sensor that transmits radio waves and receives data reflected back to the RADAR sensor to create a three-dimensional map of a scene or local area of interest.

[0086] In yet another present arrangement, imaging sensor 315 is a hyperspectral imaging device that represents a scene as multiple spectra of light. In this arrangement, different features or objects within a scene are visualized using light of specific wavelengths, and, therefore, may be better understood, analyzed, and / or visually or quantitatively described.

[0087] In preferred embodiments of the present arrangements, optional electronic display 325 displays two-dimensional or three-dimensional images based upon the type of data received from processing subsystem 313. In certain of these arrangements, optional optical assembly 330 may magnify image light received from electronic display 325, correct optical errors associated with the image light, and / or present the corrected image light to a user of optical sensor device 305.

[0088] In one embodiment of the present arrangements, IMU 342 represents an electronic device that generates fast calibration data based on measurement signals received from one or more of the position sensors 335. To this end, one or more position sensors 335 are configured to generate one or more measurement signals in response to motion of mobile device 322. If one or more of position sensors 335 provide information on locations where measurements are obtained, then in this specification, they are sometimes (e.g., in connection with Figures 10, 11, and 13) referred to as a “ground position component.”

[0089] Examples of different types of position sensors 335 include accelerometers, gyroscopes, magnetometers, another suitable type of sensor that detects motion, or a type of sensor used for error correction of IMU 342. Position sensors 335 may be located external to IMU 342, internal to the IMU 342, or some portions may be located internal to, and other portions may be located external to IMU 342. Regardless of their location, position sensors 335 may detect one or more reference points, which are used for tracking a position of mobile device 322 in a local area by using, for example, tracking module 323 of processing subsystem 313.

[0090] In processor subsystem 313, image processing engine 319 may generate, based on information received from optical sensor device 305 or from components thereof (e.g., imaging sensor 315), a three-dimensional depth mapping or multiple three-dimensional depth mappings of the space (e.g., the "scene" or the "local area" of interest) surrounding a portion or all of optical sensor device 305. In certain embodiments, image processing engine 319 of the present arrangements may generate depth information for the three-dimensional mapping of the scene based on two-dimensional information or three-dimensional information received from imaging sensor 315 that is relevant for techniques used in computing depth maps. The depth maps may include depth dimension values for each of the pixels in the depth map, which may represent multiple different portions of a scene.

[0091] Continuing with processor subsystem 313, application store 321 may store one or more software applications or programmable instruction sets for execution by processing subsystem 313 or by the optical sensor device 305. A software application may, in some examples, represent a group of programmable instructions that, when executed by a processor (e.g., processing subsystem 313 of Figure 3 or server 110 of Figure 1), generate or render content for presentation to the user. Content generated or rendered by a software application may be generated or rendered in response to inputs received from the user via movement of the optical sensor device 305 or I / O interface 317.

[0092] Examples of software applications, stored on application store 321, include gaming applications, conferencing applications, video playback applications, or programmable instructions for performing the various methods described herein. In preferred embodiments of the present arrangements, application store 321 may be a non-transitory memory store that also stores data obtained from non-imaging sensor device 340, imaging sensor 315, or from other sources included in optical sensor device 305 or received from processing subsystem 313.

[0093] Tracking module 323 may calibrate mobile device 322 using one or more calibration parameters and may adjust the calibration parameters to reduce error in determination of the position of optical sensor device 305 or of I / O interface 317. Additionally, tracking module 323 may track movements of optical sensor device 305 or of I / O interface 317 using information from imaging sensor 315, to one or more position sensors 335, IMU 342, or some combination thereof. I / O interface 317 may represent a component that allows a user to send action requests and receive responses from processing subsystem 313. In some embodiments of the present arrangements, an external controller may send such action requests and receive such responses via I / O interface 317. An action request may, in some examples, represent a request to perform a particular action. By way of example, an action request may be an instruction to start or end capture of image or video data or an instruction to perform a particular action within a software application. I / O interface 317 may include one or more input devices. Exemplary input devices include keyboard, mouse, hand-held controller, or any other suitable device for receiving action requests and communicating the action requests to processing subsystem 313.

[0094] Further, I / O interface 317 may permit mobile device 322 to interact, via a wired or wireless channel, with external devices and / or system accessories, such as additional standalonesensor systems or hand-held controllers. In preferred embodiments of the present arrangements, optical sensor device 305, processing subsystem 313, and / or I / O interface 317 may be integrated into a single housing or body. Other embodiments may include a distributed configuration of mobile device 322, in which optical sensor device 305 may be in a separate housing or enclosure, but still coupled to processing subsystem 313 by a wired or wireless communication channel. By way of example, optical sensor device 305 may be coupled to processing subsystem 313 that resides inside or is provided by an external gaming console or an external computer, such as a desktop or laptop computer. Processing subsystem 313 may also be a specialized hardware component designed to cooperate specifically with optical sensor device 305 to perform various operations described herein.

[0095] Mobile device 322 may use, among other things, a matrix of variable-phase optical elements (e.g., diffractive optical elements (DOEs)) to introduce phase delays into a wavefront of light received through a lens, thereby enhancing the performance of optical sensor device 305, or specifically one or more of imaging sensors 315 that capture aspects of a scene. These enhancements may be a function of how light passes through the variable-phase optical elementsand, in some present arrangements, may also be a function of shifting the matrix or another optical component (e.g., a sensor or a lens) of mobile device 322. By way of example, the phase delays introduced by the matrix of variable-phase optical elements may enable mobile device 322 to capture at least two different perspectives of a scene, and the different embodiments of XR systems described herein may use these different perspectives to provide or increase resolution (e.g., in an angular, depth, and / or spectral dimension) of output images or frames obtained from optical sensor device 305.

[0096] Different embodiments of an optical sensor device (e.g., optical sensor device 305 of Figure 3) disclosed herein may use variable-phase optical elements to capture different perspectives of a scene in a variety of different manners and for numerous different purposes. By way of example, a DOE may be configured to, while in an initial position, disperse light from a scene as an interference pattern on the optical sensor device, which may capture the interference pattern as a first perspective of the scene. The DOE may be shifted laterally to a subsequent position such that the resulting interference pattern represents another perspective of the scene, which may also be captured by the optical sensor device. These two perspectives may be processed to increase angular resolution (e.g., via oversampling) or to provide depth sensing (e.g., via triangulation and / or phase discrimination).

[0097] By way of example, depth values of a scene may be obtained using triangulation between two perspectives, by using a DOE to provide the two perspectives to a single optical sensor device. As another example, each element within a layer or matrix of variable-phase optical elements may be configured to deterministically phase-shift and focus light onto particular pixels (or sets of pixels) of the optical sensor device. These phase-shifted wavefronts, which may represent different perspectives of a scene, may be captured, mixed, and compared against a reference signal to detect depth within a scene.

[0098] Embodiments of the present arrangements described herein may also be implemented within various types of systems (e.g., traditional CMOS sensor systems, time-of-flight (ToF) systems, or hyperspectral imaging systems) having diverse configurations (e.g., configurations with static or movable optical components). As an example of an implementation with movable optical components (e.g., when a user chooses to move an optical sensor), the optical sensor device may include a matrix of variable-phase optical elements positioned over individual pixels or voxels of an imaging device and an actuator configured to move a component of the opticalsensor device (e.g., the matrix, a sensor or a lens) to obtain two different images representing two different instantaneous fields of view (iFOVs) per pixel. The system may then analyze these images to obtain or deduce additional spatial information for the imaged scene. In some examples with a ToF sensor, a scene may be captured in greater spatial resolution by using a conventional large pixel ToF sensor system and translating the component to oversample the portion of the image plane or scene. In examples with a non-ToF sensor (e.g., a traditional CMOS sensor), the system may perform a triangulation operation and / or a phase-discrimination operation on the different iFOVs to calculate a depth map of the scene. The system may also, for both non-ToF and ToF sensors, interpolate between the phase-shifted iFOVs to improve angular resolution of images captured by the sensors.

[0099] The oversampling process may also be used to increase spatial resolution in various hyperspectral imaging systems (e.g., snapshot hyperspectral imaging systems). Traditional hyperspectral imaging may use hyperspectral filters (e.g., tiled filters or mosaic filters) disposed directly on an imaging device to sample broadband light in the spectral domain, which may increase spectral resolution at the expense of spatial resolution. In contrast, the proposed hyperspectral imaging system may decouple the hyperspectral filters from the imaging device and position the variable-phase optical elements between the filters and the imaging device to facilitate spatial oversampling and improved spatial resolution. For example, a scene may be captured in a hyperspectral image in greater spatial resolution by translating the variable-phase optical elements to oversample portions of the image plane or scene through the individual windows of the hyperspectral filter.

[0100] In one embodiment of the present optical sensor device, each optical component is fixed in a single position and / or movable among two or more positions in a plane perpendicular to the optical axis. For example, a system with fixed optical components may introduce two or more different phase shifts in an incident wavefront. These phase-shifted signals may then be mixed and compared with a reference signal. As another example, a global shutter system may include optical elements that create two phase-shifted optical paths that are captured and stored by the imaging device while the optical elements are in a first position. The system may then shift the optical elements to a second position to create two additional phase-shifted optical paths, which may also be captured by the imaging device. As a result, the imaging device may simultaneously provide four phase-shifted signals to an electrical quadrature demodulationcomponent, where they may be mixed and compared to a reference signal to create a depth map of a scene.

[0101] Figure 4A shows a block diagram of the various software modules and an engine, each of which includes programmable instructions to carry out one or more elements, involved in the implementation of different methods according to the present teachings. A processing subsystem 413 present in a client device (e.g., processing subsystem 313 of a mobile device 322 of Figure 3) or in a server (e.g., server 110 of Figure 1) includes an image spatializing module 476, an attribute spatializing module 478, an aligning module 480 and a rendering engine 482.

[0102] Image spatializing module 476 includes, among other things, programmable instructions for spatializing a first type of pixel or voxel data to create a first type of three- dimensional spatialized data. Attribute spatializing module 478 includes, among other things, programmable instructions for spatializing a second type of three-dimensional pixel or voxel data to create a second type of three-dimensional spatialized data, which is of different type than the first type of three-dimensional spatialized data. Aligning module 480 includes, among other things, programmable instructions for aligning the first type of three-dimensional spatialized data with the second type of three-dimensional spatialized data to create an enhanced three- dimensional spatialized environment. Rendering engine 482 includes, among other things, programmable instructions for rendering, on a display component, the “processed information,” i.e., an enhanced three-dimensional spatialized environment. Although rendering engine 482 is described as a preferred embodiment, the present arrangements contemplate use of other types of engines, such as a display engine, a data engine, an execution engine, or user interface (UI) engine to achieve different results contemplated by the present teachings.

[0103] It is not necessary that each of the different modules and engines presented in Figure 4A be disposed on a processor subsystem. According to the present arrangements, these modules and engines may include a distributed configuration, in which certain modules may reside on an optical sensor device (e.g., optical sensor device 305 of Figure 3) that is coupled to a processing subsystem (e.g., processing subsystem 313, or one that resides on a smartphone or a personal computer) by a wired or wireless communication channel. In this configuration, the processing subsystem is, preferably, a specialized hardware component designed to cooperate specifically with an optical sensor and / or a non-imaging sensor to perform various operations described herein.

[0104] Figure 4B shows one exemplary distributed configuration of the modules and engine shown in Figure 4A. According to Figure 4B, a processor subsystem 413' includes attribute spatializing module 478', aligning module 480' and rendering engine 482', each of which is substantially similar to their counterparts shown in Figure 4A, i.e., attribute spatializing module 478, aligning module 480 and rendering engine 482. Processing subsystem 413' of Figure 4B is communicatively coupled via a network (e.g., The Internet) 465 to an optical sensor device 405. Further, optical sensor device 405 is similar to optical sensor device 305 of Figure 3, except that optical sensor device 405 includes an image spatializing module 476'. The programmable instructions contained inside image spatializing module 476' of Figure 4B are substantially similar to those found in image spatializing module 476 of Figure 4A. In this distributed configuration, the image spatializing module in the optical sensor device and the modules and rendering engine in the processor subsystem cooperate to carry out the present methods described herein.

[0105] Figure 5 shows an XR system 500, according to one embodiment of the present arrangements. In this embodiment, mobile device 522 includes a payload 504, which has stored therein programmable instructions 200 for carrying out certain methods of the present teachings described herein. In one embodiment of the present arrangements, mobile device 522 is substantially similar to at least one of mobile device 122 of Figure 1, mobile device 222 of Figure 2, or mobile device 322 of Figure 3. In one preferred embodiment of the present arrangements, programmable instructions 200 include an image spatializing module 476, an attribute spatializing module 478, an aligning module 480, and a rendering engine 482 of Figure 4.

[0106] Figure 6 shows an XR system 600, according to one embodiment of the present arrangements. In this embodiment, a mobile device 622, is substantially similar to at least one of mobile device 122 of Figure 1, mobile device 222 of Figure 2, or mobile device 322 of Figure 3. Payload 604 is communicatively coupled to a smartphone 618. In one aspect of this embodiment, smartphone 618 is substantially similar to smartphone 118 of Figure 1. Further, in XR system 600 of Figure 6, smartphone 618 has programmable instructions 200 stored therein for carrying out certain methods of the present teachings described herein. In an alternative embodiment of the present arrangements, modules and engines that comprise programmable instructions 200 are distributed between smartphone 618 and payload 604 of mobile device 622. In oneimplementation of this alternative embodiment, the distributed configuration of software modules is implemented between a processor subsystem that resides inside smartphone 618 and payload 604 of mobile device 622. The present arrangements recognize that programmable instructions, in a distributed configuration, are not limited to client devices.

[0107] To this end, Figure 7 shows an XR system 700, according to an alternative embodiment of the present arrangements. In this embodiment, payload 704, of mobile device 722, is communicatively coupled to a network (e.g., the Internet) 714, which has stored therein either all of or at least some of the modules or engines that comprise programmable instructions 200. Mobile device 722 is substantially similar to at least one of mobile device 122 of Figure 1, mobile device 222 of Figure 2 or mobile device 322 of Figure 3, and network 714 is substantially similar to at least one of network 114 of Figure 1, communication bus 365 of Figure 3 or network 465 of Figure 4B.

[0108] Figure 8 shows an XR system 800, according to yet another embodiment of the present arrangements. In this embodiment, payload 804, of mobile device 822, is communicatively coupled to a personal computer 816, which may function as a client device and / or a server. In one embodiment, a processing subsystem, which resides inside personal computer 816, has stored therein programmable instructions 200 for carrying out certain methods of the present teachings described herein. In an alternative implementation of the XR system shown in Figure 8, the modules and engine that includes programmable instructions 200 are distributed between personal computer 816 and mobile device 822.

[0109] Figure 9 shows an XR system 900, according to another embodiment of the present arrangements and that is a smartphone 918, which has stored therein programmable instructions 200. In this embodiment, smartphone 918 includes a display, upon which the “processed information” resulting from analysis of different types of datasets is rendered. As a result, XR system 900 of Figure 9 does not require a mobile device to perform certain methods of the present teachings described herein.

[0110] In one aspect, the present teachings offer methods for rendering a three-dimensional environment identifying the location and / or direction of location (hereinafter collectively referred to as location parameters) of one or more intangible attribute sources and / or whether an intangible attribute source is properly operating. Figure 10 shows an exemplary visual representation, presented on display 1025, resulting from a cluster analysis, according to oneembodiment of the present teachings. This figure also shows different types of signals 1002, represented in different patterns, generated from different types of intangible attribute sources (e.g., Wi-Fi™ signal generator and / or Bluetooth™ signal generator) and / or frequency regimes present in real space. Other examples of intangible attribute sources may include, but are not limited to, low voltage sources (e.g., radiofrequency (RF) transmitters, cameras, etc.) and high voltage sources (e.g., electric power generators, cellular towers, etc.).

[0111] Figure 11 shows an XR system 1100, according to one embodiment of the present arrangements. In this embodiment, a mobile device 1122 is substantially similar to at least one of mobile device 122 of Figure 1, mobile device 222 of Figure 2, or mobile device 322 of Figure 3. Mobile device 1122 includes a UAV 1102, a payload 1104, one or more non-imaging sensors 1106, and one or more imaging sensors 1108. Each non-imaging sensor 1106 captures information within its non-imaging sensor field of view 1140. Similarly, each imaging sensor 1108 captures image data within its imaging sensor field of view 1142.

[0112] As shown in Figure 11, UAV 1102 is positioned proximate to a tent 1150 such that tent 1150 is within a field of view 1140 of at least one non-imaging sensor 1106 and within a field of view 1142 of at least one imaging sensor 1106. As discussed above, a processing system (e.g., processing subsystem 313 of Figure 3) receives and processes imaging information obtained from non-imaging sensor 1106 and imaging device information obtained from imaging sensor 1108. Such processing of information allows the processing subsystem to produce the “processed information” that is rendered and / or displayed, or caused to display on display 1125.

[0113] Using the processed information, display 1125, in one embodiment of the present arrangements, shows a virtual overhead view or “bird’s eye view” the area of interest, e.g., virtual tent 1150’ and an area surrounding virtual tent 1150’. The virtual overhead view informs the user as to the location and / or direction of location of an intangible attribute source and properties of the intangible attribute source.

[0114] Display 1125 also displays multiple property demarcations 1128, e.g., concentric circles, surrounding and extend away from virtual tent 1150’ to indicate that the intangible attribute source is within tent 1150. The intangible attribute source may be one, among several, intangible attribute sources within the area of interest. As discussed above in relation to Figure 10, an area of interest may have different types of intangible attribute sources. In oneimplementation of the present arrangements, the processed information may be filtered to show one intangible attribute sources on display 1125 at a time, as shown in Figure 11.

[0115] Property demarcations 1128, on one embodiment of the present teachings, visually represent one or more properties of the intangible attribute generated by the intangible attribute source. By way of example, the number of concentric circles correlates with an intangible attribute property measurement value (e.g., signal strength value). The greater the measurement value, the number of concentric circles surround the intangible attribute source increases. Thus, a user is quickly and easily able to determine the intangible attribute source property measurement value by viewing the number of concentric circles.

[0116] By way of another example, a width or thickness of each property demarcations 1128 correlates with an intangible attribute source property measurement value (e.g., signal strength value) at a location of property demarcation. An increase in intangible attribute source property measurement value is visually represented in an increase width of a portion or all of property demarcations 1128. Returning to the Figure 11, the concentric circle closest to virtual tent 1150’ may be thicker than a concentric circle having a perimeter farther away from virtual tent 1150’ to notify the user that an intangible attribute property decreases in relation to the distance away from tent 1150.

[0117] In yet another example, one or more of property demarcations 1128 represents an outer perimeter of intangible attribute source property measurements that were obtained by one or more non-imaging sensors 1106. In other words, no intangible attribute source property measurements were obtained beyond one or more of property demarcations 1128. In this manner, a user is easily able to determine an out range or perimeter of the intangible attribute (e.g., radio signal).

[0118] Figure 12 shows an XR system 1200, according to one embodiment of the present arrangements and that displays, on a display 1225, an intangible attribute source that is located within a virtual tent 1250’, which is a virtual representation of tent 1250.

[0119] The present arrangements and teachings also allow for identification of an intangible attribute source that is hidden. XR system 1200 of Figure 12, for example, is capable of identifying and displaying an intangible attribute source hidden within a structure, e.g., a tent 1250. XR system 1200 is substantially similar to XR system 1100 of Figure 11 (i.e., mobile device 1222, UAV 1202, payload 1204, one or more non-imaging sensors 1206, one or moreimaging sensors 1208, non-imaging sensor field of view 1240, imaging sensor field of view 1242, and display 1225 are substantially similar to mobile device 1122, UAV 1102, payload 1104, one or more non-imaging sensors 1106, one or more imaging sensors 1108, non-imaging sensor field of view 1140, imaging sensor field of view 1142, and display 1125 of Figure 11).

[0120] Specifically, display 1225 displays a virtual overhead view of virtual tent 1250’ and multiple property demarcations 1228, e.g., concentric circles, have a center point located within virtual tent 1250’. A user, viewing display 1225 is able to quickly and easily identify that the location of the intangible attribute source is within virtual tent 1250’. The user knows, therefore the location of the intangible attribute source within 1250.

[0121] In one embodiment of the present arrangements, when XR system 1200 has received sufficient measurement data, from both within and exterior to tent 1250 using one or more nonimaging sensors 1206, XR system 1200 identifies the location of the intangible attribute source within tent 1250. In another embodiment of the present arrangements, XR system 1200 uses interpolating and / or extrapolation techniques, discussed in greater detail below, to identify a location of the intangible attribute source.

[0122] Figure 13 shows a virtual rendering of an area of interest 1300, according to one embodiment of the present arrangements. Display 1325 displays a virtual two-dimensional rendering of a three-dimensional area of interest 1300, which includes a virtual tent 1350’ and multiple signal strength indicators (“SSI”) 1312. Each SSI 1312 indicates a signal strength value, obtained by measurement, interpolation, or extrapolation, at that location. As a result, the rendered area of interest 1300 clearly conveys distribution information of SSIs 1312, at different locations, within area of interest 1300.

[0123] While Figure 13 conveys the distribution of signal strength of an intangible attribute source, the present arrangements are not so limited. The indicators may provide information relating other values of other intangible attribute properties (e.g., frequency regime, phase angle, wavelength, amplitude, power, data packet identifier, and data source identifier).

[0124] In one embodiment of the present teachings, animating the different renderings presented in Figure 13 allows for a user or a viewer to visualize the signal strength distribution from different perspectives, and thereby allowing the viewer or the user to identify areas of good signal strength and / or areas of week signal strength. By way of example, the user may use an I / Odevice, such as a mouse, to move to a different location within three-dimensional area of interest 1300.

[0125] In some embodiments of the present arrangements, different colors of SSI 1312 are used to convey varying signal strength values. In one embodiment, the color representation of a SSI 1312 may be presented in different colors that vary along a wavelength spectrum and preferably range from a green color representation to a red color representation. The present teachings recognize that this color-coded virtual bar representation easily conveys SSI 1312 distribution information and represents one of the many advantages of the present teachings that is not available using conventional graphical techniques.

[0126] Further, when a user selects or squeezes one selected SSI 1312, the selected SSI 1312 may be highlighted and numerical values associated with selected SSI 1312 are presented on the display to provide more concrete information about the magnitude of the RS SI measured. The present teachings recognize that SSI 1312, not only provide actual measured information, but may also provide an estimated or a predicted value of the magnitude of RSSI.

[0127] Figure 14 shows an exemplary method for determining location and / or direction of location of one or more sources of intangible attributes by rendering a two-dimensional scene, in virtual space. Figure 14 begins with element 1402, which includes obtaining, for a real space, an intangible attribute dataset (e.g., intangible attribute dataset 1500 of Figure 15) resulting from displacement in the real space of one or more vehicles e.g., one or more UAVs). Each of the mobile devices is equipped with, or communicatively coupled to one or more non-imaging sensors.

[0128] At different instances in time and at one or more of the three-dimensional coordinates that define a region or a location in the real space one or more of these non-imaging sensors measure one or more properties of one or more different types of intangible attributes present in the real space to produce the intangible attribute dataset. The intangible attribute dataset includes different instances in time of measurement, one or more three-dimensional coordinates of measurement, and one or more of the properties measured. The intangible attribute dataset, however, does not include imaging data of the real space.

[0129] Following element 1402, method 1400 of the present teachings, preferably proceeds to an element 1404 which includes obtaining an image value dataset that conveys image-related information of the real space. The image value dataset includes one or more of the three-dimensional coordinates that define the location or the region in real space of the image-related information. Moreover, image value data set includes displacing one or more the vehicle and / or an imaging sensor that is mounted on one or more the vehicles.

[0130] Following elements 1404, element 1406 includes modifying the intangible attribute dataset to produce a source identifying dataset. The source identifying dataset, in one embodiment of the present teachings, includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement, and information regarding one or more of the properties measured. The source identifying dataset does not include imaging data of the real space.

[0131] An element 1408 is carried out after element 1406. Element 1408 includes identifying, based on the source identifying dataset and without using the imaging data, the one or more of the different types of intangible attribute sources that produce one or more of the intangible attributes inside the real space.

[0132] Identifying the presence of one or more of the different types of intangible attribute sources, in one embodiment of the present teachings, includes matching at least a part of the source identifying dataset with one or more reference source identifying datasets. Each of the reference source identifying datasets is generated by and / or associated with a particular type of intangible attribute source.

[0133] Method 1400 may proceed to an element 1410, which includes determining, based on the source identifying dataset and without using the imaging, one or more locations and / or directions of locations of one or more of the identified intangible attribute sources.

[0134] Following element 1410, an element 1412 is performed. Element 1412 includes displaying and / or causing to display, in a virtual space defined inside a user interface of a user device, one or more the locations and / or directions of the locations of one or more of the different types of intangible attribute sources and properties of one or more of types of intangible attributes.

[0135] Returning to element 1402, representative intangible attributes that may exist in the real space include, but are not limited to radio signal, throughput of a connectivity signal, latency of the connectivity signal, interference of the connectivity signal, volatility of the connectivity signal, stability of the connectivity signal, RF power output, EMF, atmospheric pressure,geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, temperature, energy, force field, dielectric constant, gases, and magnetic permeability. The present teachings recognize that when the intangible attribute is, for example, energy present in real space, one or more properties of energy are capable of being measured using a set of non-imaging sensors. Examples of such properties may include, without limitation, frequency, wavelength, amplitude, signal strength (or signal power), phase angle, concentration, pressure, and phase difference. Intangible attributes may possess predefined properties depending on the intangible attribute sources generating them. For example, radio signal (e.g., radio signal 1802 of Figure 18) may include a sinusoidal wave (or a cosinusoidal wave) having a predefined amplitude and phase at different instances in time.

[0136] The intangible attribute, in on embodiment of the present teachings, is a signal propagating in the real space. The element of displaying and / or causing to display includes displaying two or more of properties of the signal propagating in the real space. The property of the signal propagating in the real space includes at least one property chosen from a group including frequency regime of the signal propagating in the real space, strength of the signal propagating in the real space, wavelength of the signal propagating in the real space, phase angle of the signal propagating in the space, phase difference of the signal propagating in the real space, angle of arrival of the signal propagating in the real space, peak frequency of the signal propagating in the real space, and direction of the signal propagating in the real space, and name of the source of the signal propagating in the real space.

[0137] In one embodiment of the present teachings, one or more of the mobile devices is an autonomous or unmanned vehicle and the user device is a processor. In another embodiment of the present arrangements, one of the mobile devices is an UAV and the user device is a laptop. In yet another embodiment, user device is an augmented and virtual reality headset, and the virtual space is a three-dimensional spatialized environment.

[0138] In one aspect of the present teachings, multiple mobile devices are displaced in the real space of interest to obtain the intangible attribute dataset. Preferably, the multiple mobile devices are displaced within the real space simultaneously. In another embodiment of the present teachings, multiple mobile devices are displaced within the real space at different periods of time.

[0139] The intangible attribute dataset, in one aspect of the present teachings, includes a three-dimensional coordinate of each non-imaging sensor and / or each imaging sensor for each measurement. As discussed above, the mobile device the mobile device may include component and / or modules for determine the three-dimensional coordinate of each measurement such as an IMU (e.g., IMU 342 of Figure 3) and one or more position sensors (e.g., one or more position sensors 335 of Figure 3), and a tracking module (e.g., tracking module 323 of Figure 3). A global positioning system (“GPS”) receiver and / or a Global Navigation Satellite System (“GNSS”) receiver may also be mounted onto of communicatively coupled to each mobile device to provide position, velocity, and timing information. Known location or position systems such as Differential GPS (“DGPS”), Real Time Kinematic (“RTK-GPS”), Integrated Internal Navigation, Visual Odometry System, Simultaneous Localization and Mapping (“SLAM”) System, Ultra-Wideband (“UWB”) Positioning System, Long-wave Infrared (“LWIR”), Infrared, and Multi-Constellation Global Navigation Satellite System (“GNNS”) system, may be used individually or in combination, to identify three-dimensional coordinate of each non-imaging sensor and / or each imaging sensor for each measurement with a high degree of accuracy.

[0140] Figure 15 shows an exemplar intangible attribute dataset 1500, according to one embodiment of the present teachings, for a radio signal resulting from one or more non-imaging sensors obtaining measurements e.g., 1502-1, 1502-2, and 1502-3) at different locations in real space, i.e., different values of X-, Y- and Z-coordinates that define real space, and measured and / or pre-stored values of intangible attribute properties of frequency, e.g., FR1, FR2, and FR3, and intensity, e.g., Il, 12, 13, 14, 15, 16 and 17, of the intangible attribute present in real space. This dataset is collected over a period of time (i.e., in a time domain) and starting from t=0 to subsequent instances in time, i.e., at t=Tl, T2, and T3.

[0141] Based on intangible attribute dataset 1500, the present teachings provide that one or more radio signal sources present in real space may produce a characteristic frequency regime, e.g., FR1, which refers to a frequency value or frequency value range, for example, ranging from fl to fl 00, wherein fl and fl 00 and the values within this range are represented by whole numbers. A radio signal source may produce one or more characteristic frequency regimes that vary spatially, i.e., with changing values of X, Y, and Z, and / or that vary temporally, i.e., with changing values of t (i.e., at t=Tl, T2 and T3). By way of example, a jamming energy source produces a characteristic frequency regime that varies spatially and not temporally. As anotherexample, a Wi-Fi energy source produces a characteristic frequency regime that varies temporally and not spatially. It may be possible that certain energy sources may produce characteristic frequency regimes that vary both temporally and spatially.

[0142] As a result, the present teachings may include a processor being configured to produce an intangible attribute dataset in the form of a spatially distributed frequency table and / or a temporally distributed frequency table. To this end, Figure 16 shows a spatially distributed frequency table 1600, according to one embodiment of the present teachings, for the characteristic frequency regime FR1 at t=Tl. Spatially distributed frequency table 1600, which is an intangible attribute dataset, according to one embodiment of the present teachings, shows that the signal strengths (or signal powers) of one or more intangible attributes at frequency regime FR1 changes as the mobile device, such as the mobile device 322 (or at least one of the nonimaging sensors mounted thereon), may be moved from one location to another.

[0143] Figure 17 shows a temporally distributed frequency table 1700, according to one embodiment of the present teachings, for the frequency regime FR1 at XI, Yl, and Z1 to develop a temporally distributed frequency table for this frequency regime. Temporally distributed frequency table 1700, which is an intangible attribute dataset, according to one embodiment of the present teachings, shows that, over time, the amplitude of frequency regime FR1 at XI, Yl, and Z1 does not change.

[0144] Figure 18 shows intangible attribute measurement graph 1800, according to one embodiment of the present teachings, generated from data from a temporally distributed reference table. In intangible attribute measurement graph 1800, radio signal 1802 is purely illustrative and shown as an example of a time domain signal that may be received by the mobile device and the processor operably connected thereto. Radio signal 1802, obtained from one of the non-imaging sensors, may be emitted or provided by one or more intangible attribute sources at a particular time or over a particular period of time within a predefined frequency range or band.

[0145] In one embodiment of the present teachings, one or more sensors obtain intangible attribute measurements (e.g., radio signal) that ranges from between about 20 kilohertz and about 75 gigahertz. In another embodiment of the present teachings, one or more sensors obtain intangible attribute measurements that ranges from between about 30 megahertz and about 6 gigahertz.

[0146] The present teachings recognize that an intangible attribute (e.g., a radio signal) may be received or obtained from one or more intangible attribute sources at the same or different instants in time. Moreover, the present teachings recognize that for the intangible attribute received or obtained in a particular frequency regime (e.g., FR1), the frequency measurements at the different X, Y, and Z-coordinates may be generated from two or more intangible attribute sources generating frequencies in this frequency regime, such that they generate: (1) overlapping frequency ranges (e.g., fl-f50 and f25-50); and / or (2) two discrete, non-overlapping, frequency ranges (e.g., fl-f50 and f75-flOO), wherein f50 and f75 are whole numbers as mentioned above

[0147] Moreover, different types of intangible attribute sources, e.g., Wi-Fi™ source, Bluetooth™ source, and / or electric power generator, may produce an intangible attribute corresponding to different frequencies and / or signal strengths. Other examples of energy sources known in the art may also be contemplated for the intangible attribute sources. Stated another way, each of the different energy sources may be typically known to produce, or operate at, their own characteristic “frequency, range of frequency, or ranges of frequencies,” which are hereinafter referred to as “frequency regimes.”

[0148] Each non-imaging sensor may be configured to measure the signal strength (or amplitude) of the received radio signal, such as the radio signal 1802, within a frequency regime (e.g., measures signal amplitude at a frequency that is between about 2401 megahertz and about 2,423 megahertz). In yet another embodiment of the present teachings, a non-imaging sensor may measure one or more properties in multiple frequency regimes. By way of example, a nonimaging sensor may measure one or more properties of an intangible attribute within a particular frequency regime and then measure, at a different period of time, one or more properties of another or the same intangible attribute within another frequency regime.

[0149] Multiple non-imaging sensors, in another embodiment of the present teachings, are utilized to measure one or more properties across a frequency regime and / or multiple frequency regimes. For example, each non-imaging sensor may measure a particular sub-range or sub-band, often referred to as a “channel,” within the frequency regime such that the entire frequency regime may be measured. By way of example, at least one of the non-imaging sensors may be configured to measure the signal strength or amplitude of a radio signal within a first frequency channel (e.g., a frequency that is between about 2401 megahertz and about 2,423 megahertz), and a second radio signal sensor measures a strength or amplitude of a signal within a secondfrequency channel (e.g., a frequency that is between about 2401 megahertz and about 2,423 megahertz). However, the present teachings are not so limited. At least one of the non-imaging sensors may be configured to measure a particular sub-range or sub-band that overlaps with another sub-range or sub-band measured by another sensor. Alternatively, the non-imaging sensors may measure some, but not all, of an intangible attribute range.

[0150] The type of frequency regimes produced from or by such energy sources may provide insight into whether an energy source is operating properly or malfunctioning. For example, malfunctioning power transformers may emit characteristic radio signals that differ from those emitted when operating as intended. As a result, the detection of such characteristic radio signals is insightful in the identification and / or a location of a malfunctioning power transformer present in a real space. Other examples may include identifying an energy source (or the intangible attribute source) in the real space based on emission of radio signals therefrom at frequency regimes (or signal strengths) different from those known or expected within a preset proximity of the mobile device.

[0151] In yet another embodiment of the present teachings, element 1402 includes obtaining a data packet that provides identifying information about an intangible attribute source.Identifying information may be selected from at least one type of information comprising medium access control (“MAC”) address, set service identifier (“SSID”) i.e., a wireless network name), data transfer rate, signal broadcast strength, broadcast frequency, unique identifier of intangible attribute source hardware, and network to which the intangible attribute source is connected.

[0152] The intangible attribute source, in one embodiment of the present teachings, acts as an access point, in that the intangible attribute source may facilitate creating a wireless local area network (WLAN) and allows for wireless-capable devices (i.e., wireless clients) to connect to the WLAN. Obtaining a data packet that provides identifying information about the intangible attribute source acting as an access point, in one embodiment of the present teachings, includes an element of transmitting, from the mobile device or a device coupled to the mobile device, a probe request management frame (hereinafter “probe request”) data packet to discover wireless networks within proximity to the mobile device or a device coupled to the mobile device. The probe request data packet may be sent passively, within one frequency range or band, or multiple frequency bands, to identify any wireless network. The probe request data packet may also besent to identify a particular wireless network having a unique service set identifier (“SSID”) The probe request data packet may include information related to supported data rates, MAC address, SSID, and wireless networking standards (e.g., an IEEE 802.11 wireless networking standard.).

[0153] Element 1402, in another embodiment of the present teachings, includes obtaining, from the intangible attribute source acting as an access point, a probe response frame (hereinafter “probe response”) data packet. The probe response data packet provides identifying information such as the MAC address, SSID of the intangible attribute source acting as an access point, supported data rates, encryption types, if required, and wireless networking standards.

[0154] The present teachings recognize that, at regular intervals (e.g., once every 0.1 seconds), access points may transmit a beacon frame data packet that may include identifying information, for example, MAC address, SSID, and / or supported data transfer rate. In another embodiment of the present teachings, obtaining a data packet that provides identifying information about the intangible attribute source acting as an access point includes an element of obtaining a beacon frame data packet from the intangible attribute source acting as an access point. As discussed above, identifying information from the beacon frame data packet may be used to generate an intangible attribute dataset.

[0155] In one embodiment of the present teachings, the intangible attribute data set is generated using the identifying information. By way of example, in Figure 15 the column header named “Property#!” may include identifying information, obtained from the data packet, such as a MAC address or a SSID instead of a frequency regime. In the present application, “attribute value” and “property value” are used interchangeably in the context of their broadest meanings. The “attribute value” may refer to a value of a property (e.g., frequency regime, signal strength, phase angle, wavelength, amplitude, power, data packet identifier, data source identifier, etc.) of an intangible attribute, such as radio signals.

[0156] In some real-world scenarios, the intangible attribute source may not be an access point but a device that is capable of connecting to or is connected to a WLAN (i.e., a wireless client). In other words, the intangible attribute source is a client e.g., a phone, camera, or audio recording device) that is connected to or is capable of connecting to a WLAN created by an access point. In one embodiment of the present teachings, the element of obtaining, from an intangible attribute source, a data packet that provides identifying information about the intangible attribute source includes obtaining the probe request data packet that is transmittedfrom an intangible attribute source acting as a client. As discussed above, the probe request data packet includes identifying information such as a MAC address or SSID.

[0157] The non-imaging sensor may be additionally configured to measure a property of one or more different types of intangible attributes at a predetermined measuring frequency (e.g., a measuring frequency of 100 measurements per second). Accordingly, the processor may store various intangible attribute properties (e.g., phase difference, wavelength, peak frequency, signal strength, etc.) and their respective values of one or more of the intangible attributes (e.g., radio signal 1802 of Figure 18) in the computer memory and create the intangible attribute dataset. The processor may determine the intangible attribute measurement at various locations (or 3D coordinates) in the real space based on a spatial movement of the mobile device therein. To minimize memory usage, processing load, and / or energy usage, however, not all of the sampled or obtained property measurements may be used in element 1402. Rather, in one embodiment of the present teachings, an optional element includes selecting one or more property measurements of one or more different types of intangible attributes at a frequency (e.g., a selecting rate of 10 measurements per second) that is less than the measuring frequency at which the sensor measures one or more of the properties.

[0158] The present teachings also recognize that one or more intangible attribute datasets for one or more real spaces may be stored in memory for use at a later time. In one embodiment of the present teachings, obtaining including obtaining or accessing a pre-stored intangible attribute dataset including one or more intangible attributes, such as radio signals, and / or various measurements related thereto.

[0159] Returning to element 1404, in one embodiment of the present teachings, a ground positioning component (e.g., position sensor 335 of Figure 3) and an imaging sensor provide or facilitate provision of an image value dataset. The systems and method obtaining the position of measurements take from non-imaging sensor, discussed in element 1402, may also be used to obtain the position of measurements taken from an imaging sensor. The ground positioning component informs on the position of the measurements taken when obtaining the image value dataset. The ground positioning component, in preferred embodiments of the present arrangements, uses relative positioning techniques, as opposed to making an absolute position measurement, as encountered when using a global positioning system (“GPS”) sensor, to arrive at the three-dimensional coordinates. Further, the three-dimensional coordinates, in thesepreferred embodiments, are obtained using positioning measurements relative to one or more reference points or coordinates. The ground positioning component may be disposed in mobile device (e.g., mobile device 222 of Figure 2 or mobile device 322 of Figure 3).

[0160] As discussed above, element 1406 includes modifying the intangible attribute dataset to produce a source identifying dataset. The source identifying dataset, in one embodiment of the present teachings, includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement, and information regarding one or more of the properties measured. The source identifying dataset does not include imaging data of the real space.

[0161] The present teachings recognize that storing, in the computer memory, and processing, using one or more processors on the mobile device or off the mobile device, one or more intangible attribute datasets may be undesirable due to the quantity of data in each intangible attribute dataset. At different locations in real space, z.e., different values of X-, Y- and Z-coordinates that define real space, the plurality of sensors may receive duplicative measured values. Therefore, the present teachings contemplate methods for filtering the intangible attribute dataset to produce a filtered intangible attribute dataset. Filtering, in one embodiment of the present teachings, includes removing, within the intangible attribute dataset, duplicative data that is spatially and / or temporally substantially similar.

[0162] By way of example, one or more duplicative data, within an intangible attribute dataset, that does not vary (within a tolerable range) spatially may be removed to reduce the size of the intangible attribute dataset. In other words, for a particular instance in time, if measurements for different locations in real space, identify the same intangible attribute properties (e.g., properties such as frequency regime or MAC address and frequency amplitude), and measure the same intangible attribute amplitude, then duplicative intangible attribute measurements may be deleted, or at least discarded from a task / data processing queue.

[0163] By way of another example, one or more duplicative data, within an intangible attribute dataset, that does not vary temporally (within a tolerable range) may be removed. For example, if the measurements for the intangible attribute property (e.g., frequency regime, MAC address, or frequency amplitude) at a particular location in real space (e.g., XI, Yl, Zl) remainsthe same at different instances in time (e.g., at t=T 1 , T2 and T3)), then duplicative data may be removed.

[0164] By way of yet another example, for a particular location of the mobile device (or the non-imaging sensors) in real space (e.g., XI, Yl, Zl), the property values of an intangible attribute may vary within the same time period or multiple time periods. Slight varying of frequency values may result due to interference from, for example, radio signals from external devices such as other electronics such as microwaves ovens, tablets, computers, computer peripherals, phones, distant routers, MRI machines, power lines, cordless telephones, smart meters. The present teachings allow for these variable measurements to be identified as a single intangible attribute, for example, a particular frequency value. In one embodiment of the present teachings, a mean is determined for all measurements of frequency band or range. The frequency regime (e.g., the signal frequency emitted by intangible attribute source) is the mean frequency value. In one embodiment of the present teachings, each measured frequency data that is within three standard deviations of the mean is replaced with the mean frequency value. In a preferred embodiment of the present teachings, each measured frequency data that is within two standard deviations of the mean is replaced with the mean frequency value. In a more preferred embodiment of the present teachings, each measured frequency data that is within one standard deviation of the mean is replaced with the mean frequency value.

[0165] Element 1406, in one embodiment of the present teachings, includes modifying the filtered intangible attribute dataset to produce the source identifying dataset. As discussed above, the source identifying dataset, in one embodiment of the present teachings, includes multiple types of information chosen from a group including information regarding the different instances in time of measurement, information regarding one or more of the three-dimensional coordinates of measurement, and information regarding one or more of the properties measured.

[0166] In one embodiment of the present teachings, data in the source identifying dataset is substantially similar to data residing in the intangible attribute dataset. For example, information regarding the different instances in time of measurement are the different instances in time the measurements were made by one or more non-imaging sensors, the information regarding one or more of the three-dimensional coordinates of measurement is the three-dimensional locations where the measurements were made by one or more of the non-imaging sensors, and theinformation regarding one or more of the properties measured is the same as properties measured by the non-imaging sensors.

[0167] In another embodiment of the present teachings, data in the source identifying dataset is different than data residing in the intangible attribute dataset. For example, information regarding one or more of the properties measured is different than one or more of the properties measured. The information regarding one or more of the properties measured, in one embodiment of the present teachings, is the result of a mathematical transformation of one or more of the properties measured.

[0168] To facilitate a mathematical transformation of one or more of the properties measured, the element of modifying the filtered intangible attribute dataset includes performing a mathematical transformation on the filtered intangible attribute dataset set to produce the source identifying dataset. Performing the mathematical transformation uses at least one transformation chosen from a group including fast Fourier transform, discrete Fourier transform, and discrete cosine transform.

[0169] In a preferred embodiment of the present teaching, a fast Fourier’s transform is used to mathematically transform the filtered intangible attribute dataset set to produce the source identifying dataset. By way of example, a fast Fourier’s transform may be performed on one or more of the properties measured and the corresponding different instances in time of measurement to produce the source identifying dataset. The information regarding one or more of the measured properties, in this implementation, is the measured property represented in a temporal frequency domain. In a preferred embodiment of the present teachings, each of the measured properties is a temporal frequency pattern, which is discussed in greater detail below.

[0170] By way of another example, a fast Fourier’s transform may also be performed on one or more of the properties measured and the corresponding one or more three-dimensional coordinates of measurement to produce a source identifying dataset in a spatial frequency domain. As a result, the source identifying dataset may provide information regarding one or more of the measured properties in the temporal frequency domain and / or the spatial frequency domain. In a preferred embodiment of the present teachings, the information regarding one or more of the properties measured, in one embodiment of the present teachings, is a spatial frequency pattern and / or a temporal frequency pattern associated with the property measured.

[0171] In one embodiment of the present teachings, the processor converts each obtained radio signal, e.g., radio signal 1802 of Figure 18, from the filtered intangible attribute dataset, into a temporal frequency domain and / or a spatial frequency domain. By way of example, one or more sensors may receive a measurement of signal strength for a first radio signal XI and a second radio signal X2 (collectively referred to as radio signals XI, X2) in time domain at a given time instant, t=Tl for a particular frequency regime, which is included in intangible attribute dataset. A processor, in modifying the intangible attribute dataset to produce the source identifying dataset may mathematically transform each of the received radio signals, such as radio signals XI and X2, into information regarding the measured property (i.e., signal strength measurement) a corresponding spatial frequency pattern and / or a temporal frequency pattern, TF1 and TF2, respectively.

[0172] Temporal frequency pattern 1900 of Figure 19, for example, is a temporal frequency pattern resulting from mathematically transforming, using a fast Fourier’s transform, radio signal 1802. Temporal frequency pattern 1900 is displayed with temporal frequency bands of 500 megahertz -1000 megahertz, 1000 megahertz -1500 megahertz, 1500 megahertz -2000 megahertz, 2000 megahertz -2500 megahertz, 500 megahertz -1500 megahertz, 1000 megahertz - 2500 megahertz, 1500 megahertz -2500 megahertz, 500 megahertz -2500 megahertz, 2500 megahertz -4000 megahertz, 500 megahertz -4000 megahertz. The number and widths of frequency bands of interest may be configured with the processor based on (i) a type of target intangible attribute source and (ii) the intended processing complexity and / or a battery usage of the processor and / or the underlying device (e.g., mobile device 322 of Figure 3). In one embodiment of the present teachings, each temporal frequency band is 80 megahertz. In yet another embodiment of the present teachings, each temporal frequency band is less than 80 megahertz.

[0173] Temporal frequency pattern 1900 includes multiple temporal frequency peaks 1902, 1904, 1906, and 1908. Each of these peaks may correspond to a temporal frequency at which the radio signal 1802 may have a signal power (or signal strength) greater than a predefined temporal frequency strength threshold value, e.g., indicated by a dashed line 1910. The processor may be configured to ignore and / or set as noise those temporal frequencies at which the radio signal 1802 may have a signal power (or signal strength) below or less than such predefined threshold value. In the illustrated example of Figure 19, the predefined threshold value is -100decibel-milliwatts. However, any other suitable predefined threshold value may be preset for the received radio signals, such as the radio signal 1802, using the processor depending on the noise level (or target intangible attribute sources) in the real space near the mobile device. In some examples, the transformed signal (in frequency domain) may include one or more frequency bands excluding a peak above the predefined threshold value.

[0174] The present teachings recognize that, at different instances in time (e.g, at t=T l, T2 and T3), the temporal frequency pattern for an intangible attribute may vary. By way of example, the temporal frequency value and / or temporal strength value for each of the one of more temporal frequency peaks, included in the temporal frequency pattern, may vary at different instances in time. Similarly, at different locations in real space, the spatial frequency pattern for an intangible attribute may vary. The spatial frequency value and / or spatial strength value for each of the one or more spatial frequency peaks included in the spatial frequency pattern, for example, may vary from one location of measurement in real space to another location.

[0175] To reduce computational complexity, memory usage and / or the amount of data in the source identifying dataset, it may be desirable to identify variable temporal frequency patterns and / or variable spatial frequency patterns as the temporal frequency pattern and / or spatial frequency of a particular intangible attribute or a particular type of intangible attribute source. The present teachings, therefore, provide an optional identifying element for identifying variable temporal frequency patterns and / or variable spatial frequency patterns as the temporal frequency pattern and / or spatial frequency of a particular intangible attribute or a particular type of intangible attribute source.

[0176] In one embodiment of the present teachings, the optional identifying element includes calculating, for an intangible attribute, a mean temporal frequency pattern from variable temporal frequency patterns. The mean temporal frequency pattern may include one or more mean temporal frequency peaks, each having a mean temporal frequency value and / or a mean temporal strength value. In one embodiment of the present teachings, each temporal frequency pattern that is within three standard deviations of the mean temporal frequency pattern (e.g., the temporal frequency value and / or temporal strength value, for one or more temporal frequency peaks of the temporal frequency pattern, is within three standard deviation of the mean temporal frequency value and / or a mean temporal strength value of the mean temporal frequency peaks of the mean temporal frequency pattern) is replaced with the mean temporal frequency pattern. In a preferredembodiment of the present teachings, each temporal frequency pattern that is within two standard deviations of the mean temporal frequency pattern is replaced with the mean temporal frequency pattern. In a more preferred embodiment of the present teachings, each temporal frequency pattern that is within one standard deviation of the mean each temporal frequency pattern is replaced with the mean temporal frequency pattern.

[0177] In another embodiment of the present teachings, for an intangible attribute having variable spatial frequency patterns, a mean spatial frequency pattern is determined. The mean spatial frequency pattern may include one or more mean spatial frequency peaks. Each mean spatial frequency peak may include a mean spatial frequency value and / or a mean spatial strength value. In one embodiment of the present teachings, each spatial frequency pattern that is within three standard deviations of the mean spatial frequency pattern (c.g., the spatial frequency value and / or spatial strength value, for one or more spatial frequency peaks of the spatial frequency pattern, is within three standard deviation of the mean spatial frequency value and / or a mean spatial strength value of the mean spatial peaks of the mean spatial frequency pattern) is replaced with the mean spatial frequency pattern. In a preferred embodiment of the present teachings, each spatial frequency pattern that is within two standard deviations of the mean spatial frequency pattern is replaced with the mean spatial frequency pattern. In a more preferred embodiment of the present teachings, each spatial frequency pattern that is within one standard deviation of the mean each spatial frequency pattern is replaced with the mean spatial frequency pattern.

[0178] Another optional element includes removing or filtering duplicative data after performing the mathematical transformation. As a result, the size of the source identifying data set may be reduced. In one embodiment of the present teachings, duplicative information (e.g, one or more spatial frequency values, one or more spatial strength values, one or more temporal frequency values, and / or temporal strength values) derived from spatial frequency patterns and / or temporal frequency patterns that are associated a particular type of intangible attribute is deleted to create the source identifying dataset. The spatial frequency patterns and / or temporal frequency patterns may be the mean spatial frequency and / or temporal frequency pattern for a particular type of intangible attribute or particular type of intangible attribute source.

[0179] By way of example, for a particular location in real space (e.g., XI, Yl, Zl), if after performing the mathematical transformation, dataset includes multiple data entries for aparticular type of intangible attribute, and if each of these includes a temporal frequency value of 1,300 Megahertz and a temporal signal strength of -70 decibel-milliwatts, then one data entry remains in the source identifying dataset and the remaining duplicative data is filtered (e.g., deleted or discarded) from a task / data processing queue. This process is commonly known as deduplication in the art.

[0180] As discussed above, element 1408 includes identifying, based on source identifying dataset and without using the imaging data, the presence of one or more of the intangible attribute sources that produce the intangible attributes inside the real space.

[0181] Identifying the presence of one or more of the intangible attribute sources, in one embodiment of the present teachings, includes matching at least a part of the source identifying dataset with one or more reference source identifying datasets. Each of the reference source identifying datasets is generated by and / or associated with a particular type of intangible attribute source.

[0182] Matching at least a part of the source identifying dataset with one or more reference source identifying datasets, in a preferred embodiment of the present teachings, includes matching, within a predefined tolerance, at least a part of the source identifying dataset with a spatial reference pattern generated by the particular type of intangible attribute source and / or a temporal reference pattern generated by the particular type of intangible attribute source.

[0183] The temporal reference pattern may include one or more reference temporal peaks and each of the reference temporal peaks has a reference temporal value. Matching at least the part of the source identifying dataset with a temporal reference pattern, in one embodiment of the present teachings, includes matching, within a predefined tolerance, at least the part of the source identifying dataset with one or more of the reference temporal values associated with the temporal reference pattern.

[0184] The spatial reference pattern includes one or more reference spatial peaks and each of the reference spatial peaks has a reference spatial value. Matching at least the part of the source identifying dataset with a spatial reference pattern, in another embodiment of the present teachings, includes matching, within the predefined tolerance, at least the part of the source identifying dataset with one or more of the reference spatial values associated with the spatial reference pattern.

[0185] The spatial reference pattern and / or the temporal pattern may be acquired by the processor for each of the particular types of intangible attribute sources sought to be identified. In other words, each type of, intangible attribute source, e.g., energy source, generates and is “associated with” a spatial reference pattern and / or the temporal pattern that may be stored in the computer memory. When at least part of the source identifying dataset matches or substantially matches a known spatial reference pattern and / or the temporal pattern, then the intangible attribute source associated with the known spatially distributed reference pattern and / or the known temporally distributed reference pattern may be identified to be present in the real space by the processor.

[0186] The present teachings recognize that, in certain embodiments, a particular type of intangible attribute generates and / or is associated with a spatial frequency reference pattern and / or a temporal frequency reference pattern. In other words, a particular type of intangible attribute generates and is associated with one or more spatial frequencies distributed in a known frequency pattern and / or one or more temporal frequencies distributed in a known frequency pattern. The spatial frequency reference pattern and / or temporal frequency reference pattern may be obtained, for example, by applying a mathematical transformation, e.g., a fast Fourier’s transform, to a signal generated by a particular type of known intangible attribute source.

[0187] To this end, element 1408, of identifying the presence of one or more of the different types of intangible attribute sources that produce one of more different types of intangible attributes inside the real space, includes matching, within a predefined tolerance, at least the part of the source identifying dataset with a spatial frequency reference pattern and / or a temporal frequency reference pattern generated by the particular type of intangible attribute source.

[0188] The spatial frequency reference pattern includes one or more reference spatial frequency peaks, each of which has a reference spatial frequency value. Additionally, the temporal frequency reference pattern includes one or more reference temporal frequency peaks, each of which has a reference temporal frequency value. Matching at least the part of the source identifying dataset with a spatial reference pattern, in another embodiment of the present teachings, therefore includes matching, within the predefined tolerance, at least the part of the source identifying dataset with one or more of the reference spatial frequency values associated with the spatial frequency reference pattern. Moreover, matching at least the part of the source identifying dataset with a temporal reference pattern, in another embodiment of the presentteachings, includes matching, within a predefined tolerance, at least the part of the source identifying dataset with one or more of the reference temporal frequency values associated with the temporal frequency reference pattern.

[0189] In yet another embodiment of the present teachings, the source identifying dataset includes one or more spatial patterns and / or one or more temporal patterns. Each of the temporal patterns includes one or more temporal peaks, each having a temporal value (e.g., 1 second) and a temporal strength value (e.g., -70 decibel-milliwatts). The temporal strength value is identified as being a strength value that is greater than a preceding temporal strength value and subsequent temporal strength value. Each of the spatial patterns includes one or more spatial peaks, each having a spatial value (e.g., 1 meter) and a spatial strength value (e.g., -70 decibel-milliwatts). The spatial strength value is identified as being a strength value that is greater than a preceding spatial strength value and subsequent spatial strength value.

[0190] Element 1408, in one embodiment of the present teachings, includes comparing, within the source identifying dataset, one or more spatial strength values of different spatial patterns and / or one more temporal strength values of different temporal patterns. Each of the spatial strength values and / or temporal strength values that are different than others of one or more of the intangible attribute strength values corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0191] In another embodiment of the present teachings, element 1408 includes determining whether one or more of the temporal values, in each of the one or more temporal patterns in the source identifying dataset, is within a predetermined target temporal band and / or one or more spatial values of one more of the spatial patterns, is within a predetermined target spatial band. Each of the temporal patterns having one more of the temporal values that are within the predetermined target temporal band and / or each of the spatial patterns having one or more of the temporal values that are within the target spatial band corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space,

[0192] In yet another embodiment of the present teachings, element 1408 includes determining whether one or more of the temporal values, of one or more of the temporal patterns match a target temporal value and / or one or more of the temporal values, of one or more of thespatial patterns, matches a target spatial value. Each of the temporal patterns having one more of the temporal values that match the target temporal value and / or each of the spatial patterns having one or more of the temporal values that match the target spatial value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0193] In yet another embodiment of the present teachings, element 1408 includes comparing, one or more temporal strength values, of one or more of the temporal patterns, to a predetermined temporal strength threshold value, and / or one or more of the spatial strength values, of one or more spatial patterns, to a predetermined spatial strength threshold value. Each of the one or more temporal patterns having one or more temporal strength values that is greater than or equal to the predetermined temporal strength threshold value and / or each of one more of spatial patterns having one or more spatial strength values that is greater than or equal to the predetermined spatial strength threshold value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0194] When two or more intangible attribute sources, present in real space, operate in frequency regimes that include overlapping frequency ranges, the source identifying dataset may be analyzed to identify the resulting two or more intangible attribute sources that may be present in an overlapping range. In preferred embodiments, the present teachings offer a differentiating technique to identify one or more frequencies in the overlapping range to determine which intangible attribute sources are generating a signal in the frequency regime.

[0195] As discussed above, in one embodiment of the present teachings, a mathematical transformation (e.g., a fast Fourier’s transform) is used to transform the filtered intangible attribute dataset set to produce a source identifying dataset in the spatial frequency domain and / or the temporal frequency domain. The source identifying dataset may include one or more spatial frequency patterns and / or one or more temporal frequency patterns. Each of the temporal frequency peaks includes a temporal frequency value (e.g., 1,300 megahertz) and a temporal frequency strength value (e.g., -70 decibel-milliwatts) that is greater than a preceding and subsequent temporal frequency strength value. Each of the spatial frequency patterns includes one or more spatial frequency peaks, each having a spatial frequency value (e.g., 1 meter-1) and a spatial frequency strength value (e.g., -70 decibel-milliwatts). The spatial frequency strengthvalue is identified as being a frequency strength value that is greater than a preceding spatial frequency strength value and subsequent spatial frequency strength value.

[0196] Element 1408, in one embodiment of the present teachings, includes comparing, within the source identifying dataset, one or more spatial frequency strength values of different spatial frequency patterns and / or one more temporal frequency strength values of different temporal frequency patterns. Each of the spatial frequency strength values and / or temporal frequency strength values that is different than other spatial frequency strength values and / or temporal frequency strength values corresponds to and are identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0197] By way of example, the source identifying dataset may include temporal frequency patterns, TF1 and TF2, that correspond to radio signals XI and X2, respectively. The processor may identify, in the temporal frequency pattern, TF1, a temporal frequency peak with the highest temporal frequency strength value, SI, and a corresponding temporal frequency value, Fl.Similarly, the processor may identify, in the temporal frequency pattern, TF2, a temporal frequency peak with the highest temporal frequency strength value, S2, and a corresponding temporal frequency value, F2.

[0198] The processor may compare the temporal frequency strength value SI and S2, of the received radio signals XI and X2, to differentiate between them. For example, if temporal frequency strength value SI is different than temporal frequency strength value S2, then temporal frequency strength value SI corresponds to and is identified as an intangible attribute source that produces radio signal XI inside the real space. Temporal frequency strength value S2 corresponds to and is identified as another or different intangible attribute source that produces radio signal X2 inside the real space.

[0199] In another embodiment of the present teachings, element 1408 includes determining whether one or more of the temporal frequency values, in each of the one or more temporal frequency patterns in the source identifying dataset, is within a predetermined target temporal frequency band and / or one or more spatial frequency values of one more of the spatial frequency patterns, is within a predetermined target spatial frequency band. Each of the temporal frequency patterns that includes one more of temporal frequency values that is within the predetermined target temporal frequency band and / or each of the spatial frequency patterns that includes one or more spatial frequency values that is within the target spatial frequency band corresponds to andis identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0200] By way of example, the processor may be configured to differentiate between the radio signals XI and X2 received at a given time instant based on a comparison between their respective temporal frequency values within a target temporal frequency band. For example, the processor may be preconfigured or dynamically configured with a target temporal frequency band ranging from 2000 megahertz to 2500 megahertz. As discussed above, the processor may identify, in the temporal frequency pattern, TF1, a temporal frequency peak with the highest temporal frequency strength value, SI, and a corresponding temporal frequency value, Fl. Similarly, the processor may identify, in the temporal frequency pattern, TF2, a temporal frequency peak with the highest temporal frequency strength value, S2, and a corresponding temporal frequency value, F2, as discussed above.

[0201] In a first example, temporal frequency value Fl may be located within the target frequency band, and the temporal frequency value F2 may be located outside the target frequency band. In this first case, the processor may be configured to select temporal frequency Fl, being located in (or belonging to) the target frequency band, as corresponding to and being identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0202] In a second example, if both the temporal frequency values Fl and F2 may be located within the target temporal frequency band, as radio signals XI and X2 are identified as intangible attribute sources that produce the intangible attributes inside the real space.

[0203] In yet another embodiment of the present teachings, element 1408 includes determining whether one or more of the temporal frequency values, of one or more of the temporal frequency patterns matches a target temporal frequency value and / or one or more of the spatial frequency values, of one or more of the spatial frequency patterns, matches a target spatial frequency value. Each of the temporal frequency patterns that includes a temporal frequency value that matches the target temporal frequency value and / or each of the spatial frequency patterns that includes a spatial frequency value that matches the target spatial frequency value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0204] By way of example, the processor may be preconfigured or dynamically configured with a target temporal frequency value of 900 megahertz. The processor compares each of the temporal frequency values Fl and F2 with the target temporal frequency value. If the temporal frequency value Fl and / or F2 match the target temporal frequency value, then the radio signal XI and / or X2, respectively, corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0205] In yet another embodiment of the present teachings, element 1408 includes comparing, one or more temporal frequency strength values, of one or more of the temporal frequency patterns, to a predetermined temporal frequency strength threshold value, and / or one or more spatial frequency strength values, of one or more spatial frequency patterns, to a predetermined spatial frequency strength threshold value. Each of one or more temporal frequency patterns that includes one or more temporal frequency strength values that is greater than or equal to the predetermined temporal frequency strength threshold value and / or each of one more of spatial frequency patterns that includes one or more spatial frequency strength values that is greater than or equal to the predetermined spatial frequency strength threshold value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0206] In yet another embodiment of the present teachings, the processor is configured to distinguish between the radio signals received in the same or differing frequency regimes based on additional properties associated with the radio signals. Examples of the additional properties may include, but are not limited to, modulation schemes, signal-to-noise ratio (SNR), bit error rate (BER), and collision detection protocols. Examples of the collision detection protocols may include, but are not limited to, carrier sense multiple access with collision avoidance (CSMA / CA) protocol and adaptive frequency hopping (AFH) protocol.

[0207] By way of example, the processor may distinguish between and / or identify the received radio signals based on an underlying modulation scheme to select only a subset of one or more radio signals, e.g., for determining the corresponding intangible attribute source. By way of another example, the processor may be configured to select only those received radio signals whose modulation parameters may pertain to a predetermined modulation scheme such as Orthogonal Frequency Division Multiplexing (OFDM) scheme. For example, the processor may determine that the received radio signals may be generated by a Wi-Fi™ source (or Wi-Fi™-enabled source) based on the received radio signals being modulated (or implementing) DSSS or OFDM scheme. In another example, the processor may determine that the received radio signals may be generated by a Bluetooth™ source (or Bluetooth™-enabled source) based on the received radio signals being modulated (or implementing) GFSK. By way of yet another example, the processor may implement any suitable time division multiplexing (TDM) algorithm known in the art to produce a multiplexed radio signal (in time domain) from the received radio signals (in time domain) having similar modulation parameters. The multiplexed radio signal may be converted into frequency domain to process the same type of radio signals as a batch for determining the highest frequency peak (and related aspects such as frequency value and frequency strength value) therein, as discussed above.

[0208] In another embodiment of the present teachings, information from a received data packet may assist the processor in identifying the presence of intangible attribute sources that produce the intangible attributes inside the real space. By way of example, a MAC address provided as part of the data packet identifies an intangible attribute source associated therewith.

[0209] Returning now to element 1410, which includes determining, based on the source identifying dataset and without using the imaging, one or more locations and / or directions of locations of one or more of the identified intangible attribute sources.

[0210] In one embodiment of the present teachings, element 1410 includes performing a clustering analysis. The goal of a clustering analysis is to determine the location and / or the direction of location of one or more of these intangible attribute sources. Clustering analysis may segregate intangible attribute measurements with similar traits into clusters. In one embodiment of the present teachings, clustering analysis analyzes the distance between one or more of the X-, Y-, and Z-coordinate values of two different clusters or the lapse of time between instances in time of detection clusters of certain frequency regimes to provide a direction of propagation in a three-dimensional space, of one or more frequencies generated by one or more intangible attribute sources. When the non-imaging sensor detects and measures this range of frequencies, and, preferably in real time, generates a responsive signal, the present teachings are able to offer the processor configured to compute a direction of propagation of a signal, such as the radio signal 1802 of Figure 16, which corresponds to the direction of the propagation of this range of frequencies and associate the direction of propagation of the signal, such as the radio signal 1802, to the particular intangible attribute source that generated this range of frequencies.

[0211] In another embodiment of the present teachings, k-means clustering analysis is used to reveal the location and / or the direction of location of each intangible attribute source. While not wishing to be bound by theory, the present teachings recognize that the processor may implement a method of / / -means clustering to determine a centroid for a predetermined number of clusters, k, wherein each centroid of each cluster is the mean of all data points within the cluster. Each cluster may include one or more data points such as signal strengths, which may correspond to (i) various radio signals and / or (ii) various frequencies in the radio signal, such as the radio signal 1802, received by the non-imaging sensors on the mobile device 322. The number of clusters,is the predetermined number of clusters that will be created during the analysis. The number of predefined clusters, in one embodiment of the present teachings, ranges from between about 2 clusters and about 10 clusters. In a preferred embodiment of the present teachings, the number of predefined clusters ranges from between about 4 clusters and about 8 clusters. In a preferred embodiment of the present teachings, the number of predefined clusters ranges from between about 5 clusters and about 7 clusters.

[0212] Determining the location and / or the direction of location of an intangible attribute source using k-means clustering, in one embodiment of the present teachings, includes the processor being configured for identifying centroids of various clusters. Each cluster may include a centroid, which may correspond to the mean signal strength in that cluster. At each position (z.e., X-, Y-, Z-coordinate values) of the mobile device 322 in the real space, the processor may compare a mean signal strength (or centroid) of each cluster with a signal strength (e.g., a temporal frequency peak having the highest temporal frequency strength value) of the radio signal 1802 (or a portion thereof) at a frequency (e.g., a temporal frequency value associated with the temporal frequency peak having the highest temporal frequency strength value). Based on the comparison, when there is a match, or a substantial match e.g., within + / - 10% of the signal strength value), between the mean signal strength and the signal strength, the centroid may indicate a location of the intangible attribute source providing the radio signal (or a portion thereof) at the frequency.

[0213] Further, as time lapses from T1 to T2 and then to T3, during different instances in time of detection of certain frequency regimes, the clustering analysis of the present teachings may include the processor being configured to incrementally analyze the signal strength measurements being obtained and compare them with the signal strength corresponding to aselected frequency (e.g., a temporal frequency peak value having the highest temporal frequency strength value).

[0214] In yet another embodiment of the present teachings, element 1410 includes performing interpolation to predict a location and / or direction of the location of intangible attribute source. Interpolating may be implemented, for example, when an intangible attribute source is disposed within a confined region (e.g., a tent, house, or building). One or more of the sensors, mounted on or communicatively coupled to a mobile device, receive intangible attribute measurements outside of the confined regions but is not able to obtain intangible attribute measurements from within the confined region. Using interpolation, the location and / or direction of the location of intangible attribute source, within the confined region, may be predicted.

[0215] In this embodiment, two or more portions of the intangible attribute dataset and / or the source identifying dataset associated with a particular type of intangible attribute source are used to compute a predicted intangible attribute measurement associated with the particular type of intangible attribute source. As discussed above, each portion of the intangible attribute dataset and / or the source identifying dataset is associated with one or more of three-dimensional location coordinates. A predicted intangible attribute measurement associated with the particular type of intangible attribute source, in one embodiment of the present teachings, is determined by calculating an intermediate positioned on a linear trajectory that intersects two or more of the corresponding three-dimensional location coordinates.

[0216] In certain aspects of the present teachings, the three-dimensional location coordinate associated with the intangible attribute source located within the confined region is relative to one or more three-dimensional coordinates that define a perimeter or boundary of the confined region.

[0217] In yet another embodiment of the present teachings, element 1410 includes performing extrapolation to predict a location and / or direction of the location of intangible attribute source. Interpolation may be implemented, for example, when an intangible attribute source is disposed outside of the real space (e.g., outside the line of sight of one or more nonimaging sensor and / or imaging sensors). Using extrapolation, the location and / or direction of the exterior location of intangible attribute source, beyond the real space, may be predicted. A predicted intangible attribute measurement associated with the particular type of intangible attribute source, in one embodiment of the present teachings, is determined by calculating anexterior positioned on a linear trajectory that intersects two or more of corresponding three- dimensional location coordinates.

[0218] Figure 20 shows, a method 2000, according to one embodiment of the present teachings of performing determining element 1410 and / or determining element 2110, discussed in greater detail below. A processor, on one embodiment of the present teachings will perform elements 2002, 2004, 2006, 2008, 2010, 2012, and 2014 for each of the intangible attribute sources, whether received simultaneously or at different instants by the non-imaging sensors. However, the description of these elements is explained herein by way of an exemplary radio signal 1802 of Figure 18 for the sake of brevity and simplicity.

[0219] Method 2000 includes an element 2002, which includes calculating, for each of the intangible attribute sources, a phase angle at a plurality of one or more non-imaging sensors. The plurality of non-imaging sensors includes a first non-imaging sensor and a second non-imaging sensor having a predetermined baseline distance there between. In one embodiment of the present teachings, a non-imaging sensor device (e.g., non-imaging sensor device 340 of Figure 3) includes the first non-imaging sensor and the second non-imaging sensor (hereinafter collectively referred to as “non-imaging sensors”) for receiving the radio signals.

[0220] In one embodiment, the processor may be configured to determine an amplitude and a difference in phase of each of the radio signals, e.g., radio signal 1802, received at each of the non-imaging sensors. The “phase” and “phase angle” are used interchangeably in the context of their broadest definitions. The “phase angle” may refer to an angular component of a signal’s waveform at an input of a receiver. The phase angle may indicate a position of the signal’s waveform along its oscillatory cycle at the moment it is received by a sensor such as the nonimaging sensor. The phase angle may represent the relative position of the waveform at a specific point in time compared to a first reference point. Examples of the first reference point may include a start of the waveform, a peak amplitude, or a zero-crossing point of the radio signal 1802. For example, the non-imaging sensors may receive the radio signal 1802 (in time domain) as shown in Equation 1. y(t) = A sin (rot -I- <p~) (1) where:y(t) = sinusoidal signal (e.g., radio signal) A = amplitude of the sinusoidal signal a> = angular frequency t = time (p = phase angle

[0221] Based on Equation 1, in one example, if a sinusoidal portion of the radio signal 1802 is received at its peak with the zero-crossing point (representing zero amplitude) as the first reference point, the processor may determine the phase angle 0 to be TI / 2 radians ( / ., 90 degrees). This means that the signal waveform is at its maximum positive amplitude at the moment the radio signal 1802 may be received at the non-imaging sensors. As such, the processor may measure a first phase angle 01, e.g., at the first non-imaging sensor, and a second phase angle 02, e.g., at the second non-imaging sensor.

[0222] Following element 2002, an element 2004 is performed on each intangible attribute received from an intangible attribute source. Element 2004 includes determining, based on the phase angle of each intangible attribute at each of the plurality of non-imaging sensors, a phase difference of each of the intangible attribute sources.

[0223] A calculation of a phase difference is represented in Equation 2.A0 = 01 - 02 (2)

[0224] Following element 2006, the method shown in Figure 20 proceeds to an element 2008. Element 2008 includes selecting, for each of the intangible attribute sources, one of one or more spatial frequency values and / or one of one or more temporal frequency values. In one embodiment of the present teachings, the selected temporal frequency value and / or spatial frequency value (hereinafter also referred to as “the selected frequency”) is the highest temporal frequency strength value and / or highest spatial frequency strength value in a temporal frequency pattern and / or spatial frequency pattern. The selected temporal frequency value and / or spatial frequency value, in another embodiment of the present teachings, is the highest temporal frequency strength value and / or highest spatial frequency strength value in a temporal frequency band and / or spatial frequency band.

[0225] By way of example, in temporal frequency pattern 1900 of Figure 19, the processor may select the temporal frequency peak 1902 within a frequency band ranging from 1000 megahertz to 1500 megahertz. The selected temporal frequency peak 1902, as illustrated, represents the temporal frequency peak, within the frequency band 1000 megahertz to 1500 megahertz, which has the highest temporal frequency strength value. The temporal frequency peak with the highest temporal frequency strength value enables the processor to select or identify the temporal frequency value (e.g., 1300 megahertz) within the frequency band (e.g., 1000 megahertz - 1500 megahertz).

[0226] Moreover, where multiple intangible attribute sources are identified in a real space, each of which generates a temporal strength value at a particular temporal frequency, the highest temporal frequency strength value may indicate that an intangible attribute source providing the corresponding radio signal (or a portion thereof) is spatially closest to the mobile device (or the non-imaging sensors) compared to any other intangible attribute sources in the real space.

[0227] Moreover, where multiple intangible attribute sources are identified in the real space, each of which generates a temporal strength value at different temporal frequencies, the highest temporal frequency strength value may indicate that an intangible attribute source providing the corresponding radio signal (or a portion thereof), is spatially closest to the AR / VR compared to any other intangible attribute sources in the real space.

[0228] The particular frequency band (e.g., 1000 megahertz - 1500 megahertz) to which the selected frequency may belong, may be selected or predefined based on a predetermined target frequency of the intangible attribute source. For example, a temporal peak and / or spatial peak may be deemed relevant and thus selected by the processor when the related temporal frequency value and / or spatial frequency value is within the predetermined or preselected frequency band of interest. In one embodiment of the present teachings, the processor is preconfigured or dynamically configured to select one or more higher frequency bands (e.g., those equal to or greater than 1000 megahertz) to identify low voltage sources. In another embodiment of the present teachings, the processor is preconfigured or dynamically configured to select one or more lower frequency bands (e.g., those less than 1000 megahertz) to identify relatively high voltage sources, in the real space proximate to the mobile device (or the non-imaging sensors). The selection of the frequency band of interest and / or the peak frequency therein may assist theprocessor in differentiating between various intangible attribute sources that may emit radio signals at different frequencies.

[0229] Next, an element 2010 includes determining, based on the selected frequency, an instantaneous wavelength of each intangible attribute source. The instantaneous wavelength of the intangible attribute (or a portion thereof) at the selected frequency may be determined using Equation 3.A = C / f (3) where:A = relative wavelength (or instantaneous wavelength) of the intangible attribute (e.g., radio signal)C = speed of light f = instantaneous frequency (or the selected frequency) of the intangible attribute

[0230] Following element 2010, an element 2012 includes calculating, for each of the intangible attribute sources, based on the baseline distance, the phase difference, and the instantaneous wavelength, an angle of arrival at each of the plurality non-imaging sensors. The angle of arrival indicates a direction in which the intangible attribute source is located.

[0231] In one embodiment of the present teachings, the processor (or the non-imaging sensors) may be configured to calculate an angle of arrival of an intangible attribute (e.g., radio signal 1802) emitted by an intangible attribute source, using Equation 4.where:A = phase difference between the intangible attribute received at each of the non-imaging sensors d = baseline distance between the non-imaging sensorsA = instantaneous wavelength of the radio signal at the selected frequency= angle of arrival of the intangible attribute at one of the non-imaging sensors (e.g., first nonimaging sensor)

[0232] In Equation 4, the angle of arrival, 01, may account for the directional sensitivity of the corresponding sensor, such as the first non-imaging sensor. The sin 0 indicates that the angle of arrival may be determined relative to a normal to a straight line (e.g., baseline distance) joining the first non-imaging sensor and the second non-imaging sensor. The angle of arrival,may indicate a direction in which an intangible attribute source may be located relative to the first non-imaging sensor, where such intangible attribute source is providing the radio signal 1802 (or a portion thereof) at the selected frequency at a given instant. Further, the processor may be configured to calculate a second angle of arrival of the radio signal 1802 at the second non-imaging sensor, using Equations 5 and 6.02= 01- arcsinwhere:A = phase difference between the radio signal received at each of the non-imaging sensors d = baseline distance between the non-imaging sensorsA = instantaneous wavelength of the radio signal at the selected frequency02= angle of arrival of the radio signal at the other non-imaging sensor (e.g., second nonimaging sensor)

[0233] In Equation 6, the angle of arrival, 02, may account for the directional sensitivity of the corresponding sensor, such as the second non-imaging sensor. The angle of arrival, 02, may be determined relative to a normal to a straight line (e.g., baseline distance) joining the first nonimaging sensor and the second non-imaging sensor. The angle of arrival, 02, may indicate a direction in which an intangible attribute source may be located relative to the second nonimaging sensor, where such intangible attribute source is providing the radio signal 1802 (or a portion thereof) at the selected frequency (e.g.., selected frequency of the selected radio signal such as radio signal 1802 at a given instant).

[0234] Following element 2012, an element 2014 is carried out. Element 2014 includes determining, for each of the intangible attribute sources, based on the angle of arrival at each ofthe plurality of non-imaging sensors and the baseline distance, a spatial location of each intangible attribute source from one of the mobile devices. The spatial location is determined relative to a current position of one of one or more of the AR / VR heads in real space.

[0235] The processor, in one example, may triangulate the spatial location of the intangible attribute source using suitable triangulation systems and / or algorithms known in the art. By way of example, the processor may obtain information received from a Light Detection and Ranging (“LIDAR”) system mounted on or communicatively coupled to the mobile device, and, using methods know in the art, the processor may determine the coordinates of the intangible attribute source. By way of another example, the processor may determine a spatial distance (SD) from the mobile device to the intangible attribute source, using Equation 7.where:SD = spatial distance to the intangible attribute source providing the radio signal (or portion thereof) at the selected frequency d = baseline distance between the first non-imaging sensor and the second non-imaging sensor= angle of arrival of the radio signal at the first non-imaging sensor02=angle of arrival of the radio signal at the second non-imaging sensor

[0236] In Equation 7, the spatial distance may refer to a normal (e.g., perpendicular distance) from the baseline distance to the intangible attribute source providing the radio signal 1802 at the selected frequency. The spatial distance may assist in determining the spatial location of the intangible attribute source with respect to the mobile device 322 (or the non-imaging sensors). For example, the processor may determine the (x, y) coordinates of the intangible attribute source, using Equations 8 and 9. x sin 0-i + y cos 9^ = d (8) x sin 92+ y cos 92— d (9) where:d = baseline distance between the first non-imaging sensor and the second non-imaging sensor 01= angle of arrival of the radio signal at the first non-imaging sensor 02= angle of arrival of the radio signal at the second non-imaging sensor

[0237] Equation 8 may represent an equation of a plane containing the first non-imaging sensor and the intangible attribute source providing the radio signal 1802 at the selected. Similarly, Equation 9 may represent an equation of a plane containing the second non-imaging sensor and the intangible attribute source providing the radio signal 1802 at the selected temporal frequency value and / or spatial frequency value. Since the values of d, 6-t , and 02may be determined, as discussed above, the processor may solve Equations 8 and 9 to determine the values of x and y corresponding to (x, y) coordinates of the intangible attribute source in the real space. Further, the processor may be configured to estimate an elevation / depression from the mobile device to the intangible attribute source providing the radio signal at the selected frequency for determining the z coordinate of the intangible attribute source in the real space. In one embodiment, the processor may be configured to determine (1) a vertical distance (or elevation) of each of the non-imaging sensors from the ground and (2) a distance to the intangible attribute source from each of the non-imaging sensors. The processor may determine vertical distances zl, z2 of the first and the second non-imaging sensors respectively using any suitable devices such as altimeters and / or rangefinders known in the art. In some examples, these altimeters and / or rangefinders may be mounted on the mobile device. Further, the processor may determine distances to the intangible attribute source from each of the non-imaging sensors, using Equations 10 and 11.where: d = baseline distance between the first non-imaging sensor and the second non-imaging sensorDI = spatial distance from the first non-imaging sensor to the intangible attribute sourceD2 = spatial distance from the second non-imaging sensor to the intangible attribute source0- = angle of arrival of the radio signal at the first non-imaging sensor02= angle of arrival of the radio signal at the second non-imaging sensor

[0238] Based on Equations 10 and 11, the processor may determine a vertical distance between the intangible attribute source and each of the non-imaging sensors, using Equations 12 and 13. vl = DI — zl (12) v2 = D2 - z2 (13) where: vl = vertical distance between the first non-imaging sensor and the intangible attribute source; v2 = vertical distance between the second non-imaging sensor and the intangible attribute source;DI = spatial distance from the first non-imaging sensor to the intangible attribute source;D2 = spatial distance from the second non-imaging sensor to the intangible attribute source; zl = vertical distance of the first non-imaging sensor from the ground; and z2 = vertical distance of the second non-imaging sensor from the ground.

[0239] Based on Equations 12 and 13, the processor may determine the elevation (or z- coordinate in the real space) of the intangible attribute source relative to the non-imaging sensors, using Equation 14. zrel= v2 - vl (14) where: zret=relative z-coordinate of the intangible attribute source in the real space

[0240] Hence, the processor may determine the location of the intangible attribute source defined by (x, y, z) coordinates in the real space based on Equations 8, 9, and 14.

[0241] The present systems and methods are not limited to the location of an energy source and may be used for detection of a gas leak. In connection with a gas leak, the present teachingsmay include the processor being configured to identify flow patterns (ambient inside a medium) emanating from a source of the gas leak.

[0242] In addition of the exemplar methods describe above, other examples of identifying the location of an object in three-dimensional space known in the art may also be contemplated for identifying an attribute source. By way of example, direct georeferencing, photogrammetric triangulation, single view metrology, ray casting with terrain model intersection, stereo vision, and structure from motion methodologies may be implemented, alone or in combination, to identify the intangible attribute source in the real space.

[0243] Returning to method 1400 of Figure 14, following element 1410, element 1412 is performed. In one embodiment of the present arrangements, the processor may be configured to modify an image and / or a video, which may be a virtual image and / or virtual video, to include at least one visual aspect selected from a group including (i) the intangible attribute source (or a location thereof) providing the radio signal at the selected frequency, (ii) one or more virtual indicators towards such location and / or a direction of such location, (iii) one or more virtual indicators away from such location and / or the direction of such location, (iv) indicate a path towards, or away from, such location and / or the direction of such location, (v) highlight or dampen a surface proximate to such location and / or the direction of such location, and (vi) remove or hide a surface (such as an object) proximate to such location and / or the direction of such location. In some examples, the image and / or the video may pertain to a computergenerated environment (i.e., a virtual space). For example, the processor may be configured to trigger a display device, in communication with the mobile device, to display the modified image (and / or the modified video). In some instances, the processor may operate the display device to display an image (and / or video) both with or without the above modification. In further instances, the processor may be configured to initiate or inhibit the computer-generated environment (i.e., a virtual space) in response to (1) such location of the intangible attribute source providing the radio signal (such as the radio signal 1802) at the selected frequency in a predetermined frequency band, (2) the direction of such location, and / or (3) the modified image and / or video. Other instances may include a processor may be configured to operate, or cease to operate, a remote device in response to the modified image and / or video.

[0244] When interpolation techniques are used in element 1410, as discussed above, element 1412 of displaying and / or the causing to display includes presenting an interior virtual object at acorresponding interior three-dimensional location coordinate associated with the particular type of the intangible attribute source. The interior virtual objects, for example, includes arrows and / or an illustration of the particular type of the intangible attribute source.

[0245] When extrapolation techniques are used in element 1410, as discussed above, element 1412 of displaying and / or the causing to display includes presenting an exterior virtual object at a corresponding interior three-dimensional location coordinate associated with the particular type of the intangible attribute source. The exterior virtual objects, for example, includes arrows and / or an illustration of the particular type of the intangible attribute source.

[0246] The exemplary method 1400 may be described in the general context of computer executable instructions. Generally, computer executable instructions may include routines, programs, objects, components, data structures, procedures, modules, functions, and like that perform particular functions or implement particular abstract data types. The computer executable instructions may be stored on a computer readable medium and installed or embedded in an appropriate device for execution. The order in which method 1400 is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined or otherwise performed in any order to implement method 1400, or an alternate method. Additionally, individual elements may be deleted from method 1400 without departing from the concepts described herein. Furthermore, method 1400 may be implemented in any suitable hardware, software, firmware, or combination thereof, that exists in the art, related art, or that is later developed. Method 1400 describes an exemplary implementation mobile device or components thereof (e.g, processing subsystem 313 of Figure 3). One having ordinary skill in the art will understand that method 1400 may be modified appropriately for implementation with other configurations and methods or by any components of the mobile device or remote devices without departing from the concepts described in the present application.

[0247] The present teachings also provide methods for rendering and / or causing to rendering a spatialized three-dimensional environment that identifies the location and / or direction of the location of one or more intangible attribute sources. The method 2100, according to one embodiment of the present teachings, preferably, begins with element 2102 including displacing one or more vehicles (e.g., aerial drone) within a three-dimensional real space, each or which is equipped with or communicatively coupled to one or more non-imaging sensors that measure, at one or more the three-dimensional coordinates that define a region or a location, one or moreproperties (e.g., frequency and / or amplitude) of one or more different types of the intangible attributes (e.g., radio signal) present in the real space. Examples of such intangible attribute properties include throughput of a connectivity signal, latency of a connectivity signal, interference of a connectivity signal, volatility of a connectivity signal, stability of a connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, or temperature.

[0248] Method 2100 further includes element 2104, which involves obtaining an unknown device dataset resulting from the displacing of one or more of the mobile devices in the three- dimensional real space, wherein the unknown device dataset includes different instances in time of measurement, one or more of the three-dimensional coordinates of measurement, one or more the properties measured, and does not include imaging data of the real space. Element 2104 of obtaining is substantially similar to the obtaining element 1402 of Figure 14 and the various embodiments described in relation to element 1402 may similarly be implemented in this element.

[0249] Following element 2104, element 2106 is carried out, which includes modifying the unknown device dataset to produce a source identifying dataset. The source identifying dataset includes multiple types of information chosen from a group comprising information regarding the different instances in time of measurement, information regarding one or more of the three- dimensional coordinates of measurement, and one or more temporal frequency patterns and / or one or more spatial frequency patterns associated with one or more the properties of one or more different types of the intangible attributes present in the real space. The source identifying data set does not include imaging data of the real space. Element 2106 of modifying is substantially similar to the modifying element 1406 of Figure 14 and the various embodiments described in relation to element 1406 may similarly be implemented in this element.

[0250] After element 2106 is carried out, element 2108 is performed, which includes identifying, based on one or more of the spatial frequency patterns and / or one or more of the temporal frequency patterns and without using the imaging data, one or more of the unknown devices that transmit the intangible attribute inside the real space. The identifying element 2108 substantially similar to the identifying element 1408 of Figure 14 and the various embodiments described in relation to element 1408 may similarly be implemented in this element.

[0251] Following the identifying of element 2108, element 2110 is performed, which includes determining, using the source identifying dataset, one or more locations and / or directions of locations of one or more unknown devices that transmit the intangible attribute inside the real space. The determining element does use any imaging data. This determining element 2110 is substantially similar to the determining element 1410 of Figure 14 and the various embodiments described in relation to element 1410 may similarly be implemented in this element.

[0252] Method 2100 further includes an element 2112, which includes obtaining an image value dataset that conveys image-related information of the real space and includes one or more of the three-dimensional coordinates, which define a location or a region in the real space of the image-related information. Element 2112 of obtaining is substantially similar to the obtaining element 1404 of Figure 14 and the various embodiments described in relation to element 1404 may similarly be implemented in this element.

[0253] In one embodiment of the present teachings, a ground positioning component e.g., position sensor 335 of Figure 3) and an imaging sensor (e.g., optical sensor device 305 of Figure 3) provide or facilitate provision of the image value dataset. The ground positioning component informs on the position of the measurements taken when obtaining the image value dataset. The ground positioning component, in preferred embodiments of the present arrangements, uses relative positioning techniques, as opposed to making an absolute position measurement, as encountered when using a global positioning system (“GPS”) sensor, to arrive at the three- dimensional coordinates. Further, the three-dimensional coordinates, in these preferred embodiments, are obtained using positioning measurements relative to one or more reference points or coordinates. The ground positioning component may be disposed in mobile device (e.g., mobile device 222 of Figure 2 or mobile device 322 of Figure 3).

[0254] The imaging sensor, like the ground positioning component, be disposed in a mobile device. If the present arrangements rely on obtaining the image value dataset from a storage medium, such as an image value database, then the storage medium may reside inside or be communicatively coupled to the mobile device so that the image value dataset information is available for processing as described below.

[0255] Next, an element 2114 is carried out, which involves spatializing, using plurality of the three-dimensional coordinates and a spatialized model, the image value dataset to create aspatialized image value dataset. Spatializing is preferably implemented using a three- dimensional spatializing model. In preferred embodiments of the present teachings, an image spatializing module (e.g., image spatializing module 476 of Figure 4A or image spatializing module 476' of Figure 4B) is used to create the spatialized image value dataset. The image spatializing module may reside on the mobile device or on an external processor. If the image spatializing module resides on the external processor, then the external processor, which is external to the mobile device, is communicatively coupled to the mobile device to convey to it the “processed information,” resulting from processing of the spatialized image value dataset.

[0256] In those embodiments where an image spatializing module is used in element 2114, this module begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the image spatializing module integrates the subdivisions to create a spatialized model of the real space. In the spatializing element 2114, the image values may be distributed, based upon the spatialized model, to create the spatialized image value dataset. Further, “real space,” in the present methods, is electronically represented using a plurality of the three-dimensional location coordinates, which may have been retrieved from obtaining elements 2104 and / or 2112.

[0257] Method 2100 further includes element 2116, which includes spatializing, using the spatialized model, the source identifying dataset to create a spatialized source identifying dataset. In preferred embodiments of the present teachings, an attribute spatializing module (e.g., attribute spatializing module 478 of Figure 4A or attribute spatializing module 478' of Figure 4B) is used to create the spatialized source identifying dataset. The attribute spatializing module may reside on the mobile device or on an external processor. If the attribute spatializing module resides on the external processor, then the external processor, which is external to the mobile device, is communicatively coupled to the mobile device to convey to it the “processed information,” resulting from processing of the spatialized source identifying dataset.

[0258] In those embodiments where an attribute spatializing module is used in element 2116, this module, like the image spatializing module of element 2114, begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the attribute spatializing module integrates the subdivisions to create a spatialized model of the real space. In spatializing element 2116, one or more of the intangible attribute properties measuredvalues may be distributed, based upon the spatialized model, to create the spatialized source identifying dataset.

[0259] In one implementation of method 2100, if a spatialized model is created using an image spatializing module or an attribute spatializing module, then the other spatializing module is free to use the created spatializing model, without the need for creating another or recreating a spatialized model to effect distribution of the data within the image value dataset or source identifying dataset, as the case may be.

[0260] After spatializing of elements 2114 and 2116, described in greater detail below, produce the spatialized intangible attribute datasets and the spatialized intangible attribute dataset and the spatialized imaging value dataset, then method 2100 proceeds to element 2118, which includes aligning the spatialized image value dataset with the spatialized source identifying dataset to create a revealing three-dimensional spatialized environment that reveals one or more locations or approximate locations of one or more of the unknown devices.

[0261] In preferred embodiments of the present teachings, an aligning module (e.g., an aligning module 480 of Figure 4A or an aligning module 480' of Figure 4B) is used to carry out aligning element 2118. The aligning module may reside on a mobile device or on an external processor, which is disposed external to and communicatively coupled to the mobile device. In method 2100, the enhanced three-dimensional spatialized environment represents the “processed information” that is subsequently rendered on a display component of a mobile device, user device, or a smartphone.

[0262] Regardless of where it is located, the aligning module may use one or more common spatial features present in both the spatialized source identifying dataset and the spatialized image value dataset to effect alignment of the two spatialized datasets. Examples of such common spatial features include vertex, longitude, or latitude. However, using one or more common vertices, found in both the spatialized source identifying dataset and the spatialized image value dataset, to effect alignment represents a preferred embodiment of element 2118.

[0263] After completing the alignment of two spatialized datasets, method 2100 proceeds to element 2120, which includes displaying and / or causing to display the three-dimensional spatialized environment identifying one or more the locations and / or directions of the locations of unknown devices and properties of one or more of the types of the intangible attributes. The displaying element uses a rendering engine. Moreover, the three-dimensional spatializedenvironment is located in a virtual space inside a user interface of a user device that is communicatively coupled to one or more of the mobile devices. Element 2120 of displaying is substantially similar to element 1412 of Figure 14 and the various embodiments described in relation to element 1412 may similarly be implemented in this element. Figures 11, 12, and 13 show exemplary images produced in virtual space from element 2120.

[0264] A rendering element (e.g., rendering engine 482 of Figure 4A or rendering engine 482’ of Figure 4B) may render the enhanced three-dimensional spatialized environment on a display component (e.g., VO interface 317 of Figure 3). In certain exemplary embodiments, the display component is integrated into a client device, which is at least one device chosen from a group comprising wearable, smartphone, and computer. In preferred embodiments of the present teachings, the wearable is a laptop.

[0265] The present teachings offer novel methods for spatializing an image dataset. In one embodiment, spatializing begins with an element that includes obtaining boundary data of a real space. In certain embodiments, this element of the present teachings includes obtaining three- dimensional coordinates that define, i.e., electronically represent, a boundary of a scene. A scene is commonly present in real space in front of a user of a user device e.g., a laptop).

[0266] Next, this method includes a subdividing element that requires subdividing the boundary data of the real space into a plurality of subdivisions. In certain embodiments, the subdividing element of the present teachings involves subdividing the electronically represented boundary obtained previously such that each of the resulting subdivisions is electronically represented by three-dimensional subdivision boundary coordinates. In other embodiments of the present teachings, the subdividing element includes spatially partitioning a real space into a plurality of subdivisions. In these embodiments, a plurality of corresponding three-dimensional location coordinates collectively forms an electronic representation of the real space, and this space undergoes spatial partitioning. The word “corresponding,” used in connection with the term “location coordinate,” conveys that the “location coordinate” is one where at least one pixelated or at least one voxelated value was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the pixelated or voxelated values obtained there.

[0267] This spatializing method then proceeds to obtaining three-dimensional pixelated data values or three-dimensional voxelated data values for corresponding location coordinates. In this element, each of the corresponding location coordinates is “associated” with at least one of theassociated three-dimensional pixelated data values or at least one associated three-dimensional voxelated data values. The word “associated,” used in connection with the terms “three- dimensional pixelated data values” and “three-dimensional voxelated data values,” conveys that the pixelated or the voxelated values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0268] In one example of this implementation, obtaining the three-dimensional pixelated data values or the three-dimensional voxelated data values includes measuring light intensity or measuring color intensity present at a unit pixel area or a unit voxel (volumetric) space, respectively, of an imaging sensor device (e.g., imaging sensor 315 of Figure 3).

[0269] According to one embodiment of the present teachings, using an optical sensor or a database to obtain the required data values may be similarly implemented.

[0270] Once the subdivisions are obtained, the spatializing method of the present teachings may proceed to an element that involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and this element generates one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisions that include at least one three-dimensional location coordinate, i.e., at least one location from where a measurement of the pixelated or the voxelated values was obtained.

[0271] Then, the spatializing method of the present teachings advances to an assigning element that includes assigning at least one of the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values to one or more of the selected subdivisions to define one or more assigned subdivisions. In this element, each of the three- dimensional pixelated data values or the associated three-dimensional voxelated data values, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated three-dimensional pixelated data values or at least one of the associated three-dimensional voxelated data values.

[0272] In one preferred aspect, the assigning element includes assigning at least one of the associated three-dimensional pixelated data values or at least one of the associated three- dimensional voxelated data values to an entire portion of the selected subdivisions. In anotherpreferred aspect, where two or more pixelated or voxelated values are available, the assigning element includes assigning a summation of weighted values of two or more of the associated three-dimensional pixelated data values or a summation of weighted values of two or more of the associated three-dimensional voxelated data values to the selected subdivisions.

[0273] After the conclusion of the assigning element, the spatializing method of the present teachings preferably proceeds to an integrating element, which involves integrating the assigned subdivisions to form a spatialized image dataset. In certain preferred embodiments, the integrating element of the present teachings includes integrating a plurality of the three- dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized image dataset. In other preferred embodiments, the integrating element of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values are distributed, based on the spatialized model, to create the spatialized image dataset.

[0274] Regardless of how they are formed, the spatialized image data set of the present teachings may be further used in other methods described herein to achieve other novel results and the spatialized image data set obtained in method 2100 represents the “processed information” that is rendered in a subsequent element on a display component of a user device, eyewear, or smartphone.

[0275] To this end, in an optional implementation, the spatializing method of the present teaching is subjected to rendering, which uses a rendering engine similar to the one described in connection with Method 2100, to render the spatialized image dataset on a display component of a client device.

[0276] In another aspect, the present teachings offer alternate novel methods for spatializing, using a spatializing model, an intangible attribute dataset. This alternate method of spatializing may begin with an obtaining element that includes obtaining boundary data of a real space. In certain embodiments of the obtaining element, three-dimensional coordinates that define, i.e., electronically represent, a boundary of a scene is obtained.

[0277] Next, the alternate method of spatializing includes a subdividing element that requires subdividing the boundary data of the real space into a plurality of subdivisions. By way of example, the subdividing element may involve subdividing the electronically representedboundary obtained in the obtaining element such that each of the resulting subdivisions is electronically represented by three-dimensional subdivision boundary coordinates. As another example, the subdividing element may include spatially partitioning the real space into a plurality of subdivisions. In this example, a plurality of corresponding three-dimensional location coordinates collectively forms an electronic representation of the real space that undergoes spatial partitioning. The word “corresponding,” used in connection with the term “location coordinate,” conveys that the “location coordinate” is one where at least one attribute value of a particular type was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the attribute value obtained there.

[0278] The alternate method of spatializing also includes another obtaining element, which requires obtaining one or more different types of attribute values for corresponding location coordinates. In this element, each of the corresponding location coordinates is “associated” with at least one type of attribute value. The word “associated,” used in connection with the terms “attribute values,” conveys that one or more attribute values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0279] In one example of this obtaining element (z.e., another obtaining element), a magnitude of an intangible property of a real space is obtained. In one implementation of this example, this obtaining element includes detecting, inside the real space, a value of a parameter, e.g., magnitude, of at least one type of attribute that is chosen from a group comprising throughput of a connectivity signal, latency of a connectivity signal, interference of a connectivity signal, volatility of a connectivity signal, stability of a connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature.

[0280] According to one embodiment of the present teachings, this obtaining element is carried out using a non-imaging sensor or a database to obtain the required data values.

[0281] Once the subdivisions are obtained, the alternate method of spatializing may proceed to an identifying element, which involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and the identifying element produces one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisionsthat include at least one three-dimensional location coordinate, i.e., at least one location from where a measurement of at least one attribute value of a particular type was obtained.

[0282] Then, the alternate method of spatializing advances to an assigning element that includes assigning at least one of the associated attribute values of at least one type to one or more of the selected subdivisions to define one or more assigned subdivisions. In this element, each of the associated attribute values of a particular type, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated attribute values of a particular type.

[0283] In one preferred aspect, the assigning element includes assigning at least one of the associated attribute values of a particular type to an entire portion of the selected subdivisions. In another preferred aspect, where two or more attribute values of a particular type are available, the assigning element includes assigning a summation of weighted values of two or more of the associated attribute values of a particular type to the selected subdivisions.

[0284] After the conclusion of the assigning element, the alternate method of spatializing preferably proceeds to an integrating element, which involves integrating the assigned subdivisions to form a spatialized intangible attribute dataset. In certain preferred embodiments, the integrating element of the present teachings includes integrating a plurality of the three- dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized intangible attribute dataset. In other preferred embodiments, the integrating element of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated attribute values of at least one type are distributed, based upon the spatialized model, to create the spatialized intangible attribute dataset.

[0285] Regardless of how they are formed, the spatialized attribute value data set of the present teachings may be further used in other methods described herein to achieve other novel results and the spatialized attribute value data set represents the “processed information” of the alternate method of spatializing and is subsequently rendered on a display component of, a user device, eyewear, or smartphone.

[0286] To this end, in an optional implementation, the alternate method of spatializing may advance to a rendering element, which uses a rendering engine, to render the spatialized intangible attribute dataset on a display component of a client device.

[0287] In yet another aspect, the present teachings offer methods for rendering an image dataset and one or more attribute values datasets.

[0288] In a preferred embodiment, this rendering method begins with an obtaining element, which includes obtaining three-dimensional pixelated data values or three-dimensional voxelated data values for one or more corresponding three-dimensional location coordinates. In this obtaining element, each of the corresponding three-dimensional location coordinates is associated with at least one associated three-dimensional pixelated data value or associated three- dimensional voxelated data value. This obtaining element is, preferably, carried out using an optical sensor and a ground positioning component.

[0289] Regardless of the methodology and components used, the rendering element provides an image of a real space (e.g., the interior and exterior of a tent). In certain embodiments of the present teachings, when such an image is taken, using an XR system of the present arrangements, metadata fields may be attached to it. These metadata fields may include at least one information chosen from a group comprising model of an imaging device, time an image was taken, whether a flash was used during image capturing, shutter speed, focal length, light value, and a location information provided by a ground positioning component. Accordingly, collections of images may be used to map out a real space such as an office environment. As will be explained below, the data pertaining to the real space is processed, according to the present teachings, to visualize the real space in virtual reality or XR from many perspectives.

[0290] Among the many perspectives, a user or a viewer is not only allowed to visualize a three-dimensional replication of the real space, but one or more users or viewers traverse the space using conventional controls such as joysticks, three-dimensional controllers, and keyboards to gather the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values described in the obtaining element. To this end, preferably, multiple client devices and controllers under the possession and / or control or users or viewers gather, as is described above in the context of “edge computing,” the associated three- dimensional pixelated data values or the associated three-dimensional voxelated data values and the corresponding location coordinates described in the obtaining element.

[0291] The rendering method also includes another obtaining element that involves obtaining, using at least one non-imaging sensor, one or more different types of attribute values for one or more of the corresponding three-dimensional location coordinates. In this obtaining element, each of the corresponding three-dimensional location coordinates is associated with at least one type of associated attribute value.

[0292] The present teachings recognize that two different obtaining elements may include many novel implementation features. By way of example, in those instances when multiple client devices and / or external controllers, such as positioning controllers, may be employed to collect different types of data values (e.g., image data values or attribute values) from a user or a viewer, the client device or the external controller, using the ground positioning component or positioning sensors, disposed thereon, aware of its relative position with respect to one or more reference points or coordinates. Moreover, the client devices or the external controllers detect their position when it is within a viewable range on a mobile application and adjusts its own location or opacity or even extends virtual controls based on its location or based on its state. When one or more users or viewers move the client device or the external controller away from the reference point(s) or coordinate(s), the display might ultimately present the same image data and / or the intangible property data for the real space, but the “processed information” may be rendered from a different viewpoint and / or may be rendered from a perspective that is further away in distance.

[0293] In the two different obtaining elements, and even certain subsequent elements, conventional three-dimensional mapping techniques may be employed to effectuate certain embodiments disclosed herein. By way of example, different types of datasets may employ different modeling technologies to optimize data visualization and three-dimensional interactivity. Moreover, certain dataset animations may be simulated by rapidly plotting datasets in succession.

[0294] As another example, graphical user interface elements may be anchored to a three- dimensional visualization or rendering that presents a state of the selected data analytics. For example, and without limitation, a slider could be mapped to an opacity of a data layer (i.e., corresponds to a dataset of a particular type). As the slider is moved up, the data layer threshold level of the particular type may change and the data layer becomes more opaque; conversely, as the slider is moved down, the data layer of the particular type becomes more transparent. Thesegraphical elements may animate for better usability, such as fading in text or moving arrows and / or lines. In the slider layer example, the actual opacity value and / or a percentage may be drawn on the slider knob itself as it is being slid, and then disappear after a predetermined duration, e.g., about 2 seconds, from a time when the user had finished moving the slider. Thus, the graphical interface elements of the present teachings are capable of conveying both, a state of one or more different types of data, as well as, a state of a client device’s or an external controller’s buttons and relevant components.

[0295] The present teachings recognize that correlating image data with intangible property data allows XR systems of the present arrangements to present a multi-variable view of a physical space in a single rendering.

[0296] After the obtaining element, mentioned above, has obtained a requisite number of pixelated or voxelated values, the rendering method may proceed to a spatializing element, which requires spatializing the three-dimensional pixelated data values or the three-dimensional voxelated data values to create a spatialized image dataset. In preferred embodiments, the spatialized element relies upon a plurality of the corresponding three-dimensional location coordinates to arrive at the spatialized image dataset. By way of example, one or more elements described in the rendering method may be carried out to create the spatialized image dataset.

[0297] After the rendering method has obtained a requisite number of attribute values, it may proceed to a spatializing element, which requires spatializing the associated attribute values of at least one particular type to create a spatialized intangible attribute dataset of at least that type. This spatializing element, like the above-mentioned spatializing element, may rely upon a plurality of the corresponding three-dimensional location coordinates to arrive at the spatialized intangible attribute dataset.

[0298] After a spatialized image dataset and a spatialized intangible attribute dataset are obtained, the rendering method proceeds to an aligning element, which includes aligning the image spatialized dataset with the attribute value spatialized dataset to create an enhanced three- dimensional spatialized environment. By way of example, one or more common features, e.g., vertex or vertices, longitude, or latitude, present in both the image spatialized dataset and spatialized intangible attribute dataset, are used to effect alignment. The enhanced three- dimensional spatialized environment of the aligning element represents the “processedinformation” of the rendering method, and in a subsequent element is rendered on a display component of a user device, an eyewear, or a smartphone.

[0299] As a result, the rendering method then proceeds to a rendering element, which involves rendering, using a rendering engine and on a display component, the enhanced three- dimensional spatialized environment.

[0300] Each of Method 1400 of Figure 14 and method 2100 of Figure 21 represent different types of programmable instruction of the present teachings that reside on XR systems of the present arrangements. The present teachings recognize that, although use of XR systems of the present arrangements represent a preferred mode of implementation, these programmable instructions may be implemented using other structural details not described herein.

[0301] In one embodiment, programmable instructions of the present teachings, rely on complex data spatialization, sampling algorithms, and machine learning. When the different present methods described herein are implemented as a software application, such applications discover signals emanating from nearby communication devices, which rely on radio waves for wireless connectivity, e.g., Wi-Fi™ routers and Bluetooth™ devices. This information is preferably displayed back in a fully interactive, completely immersive overlay shown through a display of a present XR system, like a headset or smartphone.

[0302] The present teachings recognize that a distinction may be made between a signal collection device and a signal visualization device, even though they may be the same device in certain embodiments of the present teachings. A signal collection device, commonly called a sensor, is classified as one possessing a measurement device. In the context of radio frequency data, this would be a device containing an RF receiver and antenna operating on protocols like Wi-Fi™, Bluetooth™, ZigBee™, Z-Wave™, 3G™, 4G™, LTE™ and 5G™. A signal visualization device, on the other hand, is typically characterized by a display capable of showing objects mapped into real space. Visualization may include the use of holographic elements to create the perception of objects floating in front of the user, as in the case of AR headsets or smartphones. VR headsets may use simulated environments that allow remote viewing of a real space. Mixed Reality (“MR”) and / or XR devices are typically a combination of both technologies, i.e., AR and VR, and allow for either type of experience.

[0303] Examples of devices that include both signal collection and signal visualization components include HoloLens™ available from Microsoft™, Magic Leap One™ available fromMagic Leap™, and smartphones like iPhone X™ available from Apple™, Inc., and Galaxy S10™ available from Samsung™. Examples of devices that include only signal collection components include specialized RF antenna, microphone, and Geiger counter. Examples of devices that include only signal visualization components include Meta™ 2 (z.e., an AR headset), Oculus Rift™ (i.e., VR headset), and HTC Vive™ (z.e., MR headset).

[0304] In one embodiment of the present arrangements, signal measurements are accomplished by mounting many kinds of sensors to the signal collection device, and these sensors are discussed in connection with Figures 2 and 3 described herein.

[0305] The present teachings recognize that by using sensor fusion and implicit directionality from the mobile device, triangulation of signals is possible. Further, by using headset-mounted or standalone microphones, advanced audio profiles may be built of the real space that may model sources of noise as well as noise propagation characteristics like the reverberation factor of the materials. The present teachings further recognize that using special high-pass and low-pass filters allows triangulation of certain types of equipment, like low-humming industrial generators, or high-pitched electrical transformers. Even in a typical home, devices like televisions, computers, and wireless routers emit distinct patterns of noise that is beyond the human ear’s range but within the microphone’s range. Such recognition allows the present teachings to accomplish audio fingerprinting. Representative devices used for audio fingerprinting include at least one device chosen from a group comprising thermometer, hygrometer, barometer, and airflow sensor. In certain embodiments, the present teachings recognize that the use of simple climate instruments allows measurement of and ultimately visualization of signals in 3D to accomplish such things as HVAC mapping, microclimate analysis, chemical tracing, insulation planning, and food storage auditing.

[0306] The sensors used in the present arrangement may be used to detect radiation, i.e., both thermal and infrared. To this end, examples of such sensors include Geiger counter and specific isotope detectors used in laboratories or nuclear plant settings.

[0307] In other embodiments, sensors in the present arrangements include those that detect certain harmful gases, e.g., carbon monoxide, methane, and natural gas, and naturally occurring but not always easily detected harmful mold. In this context, the present teachings recognize that sensors that detect smoke may be useful when implementing methods for informing regarding certain conditions. Smoke detecting sensors are useful for emergency scenarios like firefightingand industrial safety, or ventilation planning. Similarly, light detection sensors used in the present arrangement may be used in the form of a light meter to collect accurate color intensity and temperature values from the environment. This information may be useful to the film, lighting, and interior design industries to map out the reflectivity and general photographic atmosphere of the scanned scape.

[0308] In those instances, where vibratory information is deemed useful or necessary, sensors in the present arrangement are those that detect vibrations and seismic activity. For home automation applications, the sensors of the present arrangement detect overlay locations of home appliances, HVAC controls, light switches, and power outlets. Further, based on image recognition of objects, manual entry, or triangulation of beacons, the present teachings are applied to accomplish Industrial Automation.

[0309] Regardless of whether the need is for automation, the present arrangements preferably include components that facilitate spatial data collection. To this end, the collection process preferably requires one or more users to take sensor signal measurements from various locations inside a space. More preferably, these sensor signal measurements are evenly distributed in the target space. By way of example, when the present teachings are implemented as a software application on a mobile device (e. , eyewear or smartphone), sensor signal measurements may be automatically made while one or more users walk around the space. These users may either mount a sensor to the headset or AR- or XR-capable display, hold out a signal collection device within view of the headset or AR-or XR-capable display, or both.

[0310] Upon activating the signal collection mode, one or more of such users then follow onscreen instructions prompting them to walk around the area and scan it into the system. This may also be referred to as an active mode of operation. The information resulting from the active mode of operation may be stored and / or processed in the cloud or on a local cluster of graphic processing units (“GPUs”).

[0311] Before, during, or after the collection process, a calibration procedure may be invoked that requires collecting the location of certain types of sensors. By way of example, for calibration of a radio frequency (RF) antenna, a user may be instructed to precisely locate, within a real space, a known signal emitter, such as a Wi-Fi™ router, through a headset (e.g., AR or XR headset). The user may then be instructed to clockwise walk around the known signal emitter,keeping it at about an arm’s length, and then be instructed to counter-clockwise walk around the same emitter.

[0312] During this procedure, the visualizing system or spatializing subsystem collects hundreds or thousands of sensor data samples at certain locations in space, while visually tracking the emitter. The system then performs a mathematical analysis on the measurements and the precise location of those measurements to model and build a 3D representation of the sensor’s signal measurements. By identifying the location of the emitter, the visualizing system or spatializing subsystem may lock onto the emitter’s exact location and then measure variations in the sensor’s signal measurement readings during the user’s circular motion. Assuming a constant emitter output and that the emitter’s signal output should remain relatively constant along a circular path around the emitter, any variations in signal collection may be attributed, with a higher degree of certainty, to the sensor’s performance and dynamics.

[0313] Cellular antennas, for example, do not emit radiation in a perfectly spherical shape, but rather in multiple overlapping elliptical shapes that may very well not be symmetrical. In this case, the present teachings recognize that carrying a protective case around the signal collection equipment, or holding it in a certain way, may also alter the antenna’s performance.

[0314] The present arrangements recognize that more than one antenna may be used during collection of sensor signal measurement. The collection process of the present teachings, in certain instances, supports multiple simultaneous users, given that they have a networked connection enabling the many users’ applications (software embodying the present teachings and installed on a client device) to communicate. When operating in multi-user collection mode, the real space is subdivided into partitions and each user is responsible for scanning their respective area, which falls within a particular partition. In the case that the two areas overlap, then the measurements from those overlapping areas may be used to improve calibration and accuracy because of the proximity of the sensor readings.

[0315] There are two modes of collection of sensor signal measurements, i.e., active mode of collection and passive mode of collection. In the active mode of collection, the user is shown a scanned outline of the space and visually directed (for example, by presenting arrows in the user’s AR or XR headset) to walk to various points for taking sensor signal measurements. Feedback representing a measurement of the sensor signal is shown on the headset as collection from different areas within a space are completed. In the passive mode of collection, on the otherhand, an application (software embodying the present teachings and installed on a client device) is capable of running as a background process, “listening” for sensor signals while the user is performing another task or even playing a game. Most such applications installed on spatial computing devices, such as AR or XR headsets, continuously utilize these sensor signal scanning features. In the passive mode of collection, while other tasks are being carried out in the foreground, the sensor signal scanning for collection purposes is taking place in the background for later processing. Regardless of whether the active mode of collection or the passive mode of collection is being implemented, the collection element or process collects not only signal measurement data, but also the coordinates of the location where that signal measurement was obtained. This spatial data may be stored, in an encrypted format, if necessary, for security purposes, in a memory (for example, located on a local device with a database or log) from where it is later retrieved for processing. The spatial data may alternatively be stored on a remote server or using cloud-based synchronization. As yet another example, the present teachings may rely upon local caching of cloud data. The present teachings recognize that it is possible to retrieve existing real space measurements, which originated from other users or public or private building plans, from cloud storage.

[0316] The signal measurement data and its corresponding location coordinates may be used to perform edge and cloud GPU signal analysis, interpolation between one or more sensor signal measurements, and extrapolation of sensor signal measurement data, using algorithms and calculations per signal type, to predict sensor signal values outside a space. In certain instances, the present arrangement includes employing dedicated hardware on-device for single pass filtering and live feedback, sync to the cloud for supercomputer-class multi-pass filtering and far more advanced algorithmic processing. As mentioned above, cloud storage may also aggregate multiple users’ sensor signal measurements to fill in the gaps. For processing sensor signal data, the present teachings may rely upon CPU-based raytracing or any traditional approach of raytracing. In one embodiment, the present teachings use multiple cores by distributing equal parts of a scene or space that is being perceived by the user’s AR or VR headset. In an alternate embodiment, the present teachings use the relatively new techniques of GPU-based raytracing that perform raytracing like CPUs in the traditional approach. By way of example, when live GPU raytracing is implemented, the present teachings recognize that special GPU (repurposing game technology) may be implemented. To this end, Nvidia’s RTX hardware, which includesbuilt-in special raytracing instructions, works well. An application (software embodying the present teachings and installed on a client device) may translate the sensor signal measurements into a format the GPU is able to recognize and understand, so that the GPU is effectively fooled into thinking it is rendering a video game scene.

[0317] The present teachings may also rely upon mobile raytracing that will enable highly accurate results even when running on battery power in the field. During such a process, the present teachings may implement such elements as storing processed data, rendering of VR, AR, or XR signals, and / or providing feedback during the sensor signal measurement collection process.

[0318] In those instances, where the present teachings rely upon one or more users to walk around the real space, the general shape of the sensor signal measured may be recorded and mapped into a spatial model by an underlying AR / VR / XR operating system of the present arrangements. This three-dimensional model or spatialized model may be made available to the system for building an approximate virtual representation of the real space’s layout and contents. This is electronically represented to the user as a “mesh,” which is overlaid on top of the detected shapes. The present system may use a similar concept for showing mapped and unmapped areas during collection of sensor signal measurements. The present teachings recognize that when one or more people walk more of the different locations inside the space and collect attribute values more values for the different locations resulting the three-dimensional spatialized attribute value data of higher resolution, and this data is continuously updating as more people are walking to cover more area - in other words, the participants contribute to the higher resolution of the three- dimensional spatialized attribute value data).

[0319] In one aspect of the present teachings, unscanned, unmeshed areas are represented by dim, semi-transparent grids of dots / vertices that may protrude into as-yet-unmeshed elements of the real space. In another aspect of the present teachings, unscanned, meshed areas are represented by dim, semi-transparent grids of dashes or connected lines that roughly approximate the shape of the real space and its contents. In yet another aspect of the present teachings, scanned, meshed areas are shown with bold, colored vertical or horizontal lines with a brightness corresponding to the confidence of the signal measurements. In certain of these visual configurations, the present arrangements may display areas that have not been scanned, or display areas that have been scanned but are outdated, or have low confidence, or display areasthat have not been scanned and may be interpolated or extrapolated, and as a result, do not need to be scanned.

[0320] In certain embodiments of the present teachings, a raw signal may be represented in the background or taking the form of “fog” and a processed signal may be represented in the background of or taking the form of “rain.” When rendering the enhanced three-dimensional spatialized environment, the present teachings may show metadata like protocol versions, device addresses, and frequencies. Other examples presented during rendering include displaying techniques for extreme-scale including instancing, point clouds, and frustum culling. In the context of a rendered enhanced three-dimensional spatialized environment, the present arrangements may allow for at least one of: interacting with a signal representation by a user; toggling between signal layers; using a graphical user interface to enable / disable multiple signal overlays; proximity-based control; activating whichever signal layers are the strongest or have the most accurate data; hand interaction for precise measurements in space; holding out hand to get extrapolated measurement at that exact point; touching the interactive signal in the form of “raindrops” to get detailed information; controlling time for historical analysis; performing transceiver activity overlay; displaying emitters; animating 3D icons are placed at the location of detected transmitters; displaying sensors and signal collectors, displaying signals passing between devices; animating lines of various styles including dotted and dashed are drawn between transmitters and receivers; and / or displaying the direction and general path of that signal through the air. In the case that the present system detects physical barriers and signal reflections, the lines shown will follow the model to create the most accurate display. By way of example, line segments use a combination of color, size, and frequency of animation to indicate the magnitude of signal activity such as bandwidth or latency of network communication.

[0321] In those embodiments when the present teachings rely upon a multi-user operation, fusion of sensor signal measurements obtained from multiple signal collectors on multiple users is performed. When merging or fusing sensor signal measurements obtained from multiple signal collectors, the present teachings preferably use sensor proximity as a factor in merging of fusing the sensor measurement readings into the collection process. To this end, the above-mentioned location calibration process facilitates accurately fusing sensor measurement data. By way of example, fusing or merging, according to the present teachings, relies upon a process that is similar to that performed by differential GPS systems, which uses known points for performingcertain corrections. The present teaching, when implementing fusing or merging, may also use AI / ML for transceiver placement and real space-scale optimization. In other embodiments, the present teachings may carry out macro-scale aggregation for carrier-level optimization to accomplish fusing or merging as described herein.

[0322] Although the invention is illustrated and described herein as embodied in one or more specific examples, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit of the invention and within the scope and range of equivalents of the claims. Accordingly, it is appropriate that the appended claims be construed broadly, and in a manner consistent with the scope of the invention, as set forth in the following claims.

Claims

CL IMSWhat is claimed is1. A method for determining location and / or direction of location of one or more sources of intangible attributes, said method comprising: obtaining, for a real space, an intangible attribute dataset resulting from displacement in the real space of one or more vehicles, each of which is equipped with or communicatively coupled to one or more non-imaging sensors measuring, at different instances in time and at one or more three-dimensional coordinates that define a region or a location, one or more properties of one or more different types of said intangible attributes present in said real space, wherein said intangible attribute dataset includes different instances in time of measurement, one or more said three-dimensional coordinates of the measurement, one or more of said properties measured, and does not include imaging data of said real space; obtaining an image value dataset that conveys image-related information of said real space and includes one or more of said three-dimensional coordinates, which define said location or said region in said real space of said image-related information, wherein said obtaining said image value dataset includes displacing one or more said vehicle and / or an imaging sensor that is equipped with or communicatively coupled to one or more said vehicles; modifying said intangible attribute dataset to produce a source identifying dataset, which includes multiple types of information chosen from a group comprising information regarding said different instances in time of measurement, information regarding one or more of said three- dimensional coordinates of measurement, and information regarding one or more of said properties measured, and does not include imaging data of said real space; identifying, based on said source identifying dataset and without using said imaging data, one or more of said different types of said intangible attribute sources that produce one or more different types of said intangible attributes present inside said real space; determining, based on said source identifying dataset and without using said imaging data, one or more locations and / or directions of locations of said one or more of said type of said intangible attribute sources; and displaying and / or causing to display, in a virtual space defined inside a user interface of a user device that is communicatively coupled to one or more of said vehicles, one or more said locations and / or said directions of said locations of one or more of said different types of saidintangible attribute sources and properties of one or more of said types of said intangible attributes.

2. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein one of said vehicles is an unmanned vehicle and said user device is a processor.

3. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 2, wherein one of said vehicles is an aerial drone and said user device is a laptop.

4. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said user device is an augmented and virtual reality headset and said virtual space is a three-dimensional spatialized environment.

5. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said intangible attribute is a signal propagating in said real space, wherein said displaying and / or causing to display includes displaying two or more of said properties of said signal propagating in said real space, and wherein said property of said signal propagating in said real space includes at least one property chosen from a group comprising, frequency regime of said signal propagating in said real space, strength of said signal propagating in said real space, wavelength of said signal propagating in said real space, phase angle of said signal propagating in said real space, phase difference of said signal propagating in said real space, angle of arrival of said signal propagating in said real space, peak frequency of said signal propagating in said real space, and direction of said signal propagating in said real space, and name of said source of said signal propagating in said real space.

6. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said determining one or more locations and / or directions of locations of said one or more of said different types of said intangible attribute sources includes interpolating, using two or more portions of said intangible attribute dataset and / or said source identifying dataset associated with a particular type of said intangible attribute source and with two or more corresponding three-dimensional location coordinates, to compute a predicted intangible attribute measurement associated with said particular type of said intangible attribute source at a corresponding interior three-dimensional location coordinate that is disposed inside a confinedregion within said real space and positioned on a linear trajectory that intersects two or more of said corresponding three-dimensional location coordinates, and wherein said real space being defined using said two or more of said corresponding three-dimensional location coordinates; and wherein said displaying and / or said causing to display presents an interior virtual object at a corresponding interior three-dimensional location coordinate associated with said particular type of said intangible attribute source and said interior virtual objects includes arrows and / or an illustration of said particular type of said intangible attribute source.

7. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 6, wherein said corresponding interior three-dimensional location coordinate associated with said particular type of said intangible attribute source is relative to one or more three-dimensional coordinates that define a perimeter or boundary of said confined region.

8. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said determining one or more locations and / or directions of locations of said one or more of said type of said intangible attribute sources includes extrapolating, using two or more portions of said intangible attribute dataset and / or said source identifying dataset associated with a particular type of said intangible attribute source and with two or more corresponding three- dimensional location coordinates, to compute a predicted intangible attribute measurement associated with said particular type of said intangible attribute source at a corresponding exterior three-dimensional location coordinate that is disposed outside said real space and positioned on a linear trajectory that intersects two or more of said corresponding three-dimensional location coordinates, and wherein said real space being defined using two or more of said corresponding three-dimensional location coordinates; and wherein said displaying and / or said causing to display presents an exterior virtual object at a corresponding exterior three-dimensional location coordinate associated with said particular type of said intangible attribute source and said exterior virtual objects includes arrows and / or an illustration of said particular type of said intangible attribute source.

9. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein in said obtaining, one or more of said properties ofone or more different types of said intangible attributes is at least one property chosen from a group comprising frequency regime, wavelength, amplitude, signal strength, phase angle, concentration, pressure, and phase difference.

10. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein in said obtaining, said intangible attribute is at least one attribute chosen from a group comprising radio signal, throughput of a connectivity signal, latency of said connectivity signal, interference of said connectivity signal, volatility of said connectivity signal, stability of said connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature.

11. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said modifying includes performing a mathematical transformation on said intangible attribute dataset set to produce said source identifying dataset, wherein said performing said mathematical transformation using at least one transformation chosen from a group comprising fast Fourier transform, discrete Fourier transform, and discrete cosine transform.

12. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 10, wherein said modifying further comprises: filtering said intangible attribute dataset to produce a filtered intangible attribute dataset wherein said filtering removes, within said intangible attribute dataset, data that is spatially and / or temporally substantially similar, wherein said filtering is carried out after said performing said mathematical transformation; and wherein said modifying includes modifying said filtered intangible attribute dataset to produce said source identifying dataset.

13. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said identifying includes matching at least a part of said source identifying dataset with one or more reference source identifying datasets, wherein one or more of said reference source identifying datasets is generated by a particular type of intangible attribute source.

14. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 13, wherein said identifying includes matching, within apredefined tolerance, at least said part of said source identifying dataset with a spatial reference pattern generated by said particular type of said intangible attribute source and / or a temporal reference pattern generated by said particular type of intangible attribute source; wherein said spatial reference pattern includes one or more reference spatial peaks, each of said reference spatial peaks having a reference spatial value; wherein said temporal reference pattern includes one or more reference temporal peaks, each of said reference temporal peaks having a reference temporal value; and wherein said matching includes matching, within said predefined tolerance, at least said part of said source identifying dataset with one or more of said reference spatial values associated with said spatial reference pattern and / or with one or more of said reference temporal values associated with said temporal reference pattern.

15. The method for determining location and / or direction of location of one or more sources of intangible attributes of claim 1, wherein said displaying and / or said causing to display further comprising: spatializing said source identifying dataset to create a spatialized intangible attribute dataset; spatializing said image value dataset to create a spatialized image value dataset; aligning said spatialized image value dataset with said spatialized intangible attribute dataset to create a revealing three-dimensional spatialized environment that reveals one or more locations or approximate locations of one or more of said intangible attribute sources; and rendering and / or causing to rendering, using a rendering engine and in said virtual space inside said user interface of said user device, said three-dimensional spatialized environment identifying one or more said locations and / or said directions of said locations of one or more of said different types of said intangible attribute sources.**** Second Independent Claim ****16. A method for determining location and / or direction of location of one or more sources of unknown intangible attributes (“unknown devices”), said method comprising: displacing one or more vehicles within a three-dimensional real space, each or which is equipped with or communicatively coupled to one or more non-imaging sensors that measure, atone or more three-dimensional coordinates that define a region or a location, one or more properties of one or more different types of said intangible attributes present in said real space, obtaining an unknown device dataset resulting from said displacing of one or more of said vehicles in said three-dimensional real space, wherein said unknown device dataset includes different instances in time of measurement, one or more of said three-dimensional coordinates of measurement, one or more said properties measured, and does not include imaging data of said real space; modifying said unknown device dataset to produce a source identifying dataset, which includes multiple types of information chosen from a group comprising information regarding said different instances in time of measurement, information regarding one or more of said three- dimensional coordinates of measurement, and one or more temporal frequency patterns and / or one or more spatial frequency patterns associated with one or more said properties of one or more different types of said intangible attributes present in said real space, and does not include imaging data of said real space; identifying, based on said one or more of said spatial frequency patterns and / or one or more of said temporal frequency patterns and without using said imaging data, one or more of said unknown devices that transmit said intangible attribute inside said real space; determining, using said source identifying dataset and without using said imaging data, one or more locations and / or directions of locations of said one or more of said unknown devices and / or frequency and / or strength of one or more different types of said intangible attributes of one or more of said unknown devices; obtaining an image value dataset that conveys image-related information of said real space and includes one or more of said three-dimensional coordinates, which define a location or a region in said real space of said image-related information; spatializing, using plurality of said three-dimensional coordinates and a spatialized model, said image value dataset to create a spatialized image value dataset; spatializing, using said spatialized model, said source identifying dataset to create a spatialized source identifying dataset; aligning said spatialized image value dataset with said spatialized source identifying dataset to create a revealing three-dimensional spatialized environment that reveals one or more locations or approximate locations of one or more of said unknown devices; anddisplaying and / or causing to display, using a rendering engine and in a virtual space inside a user interface of a user device that is communicatively coupled to one or more of said vehicles, said three-dimensional spatialized environment identifying one or more said locations and / or said directions of said locations of unknown devices and properties of one or more of said types of said intangible attributes.

17. The method for determining location and / or direction of location of one or more unknown devices of claim 16, wherein each of one or more of said spatial frequency patterns includes one or more spatial frequency peaks, each of which includes a spatial value and a spatial strength value and / or each of one or more of said temporal frequency patterns includes one or more temporal frequency peaks, each of which includes a temporal frequency value and a temporal frequency strength value; wherein said identifying includes differentiating, using said source identifying dataset, at least one differentiating method chosen from a group comprising: comparing, within said source identifying dataset, one or more spatial frequency strength values of different spatial frequency patterns and / or one more temporal frequency strength values of different temporal frequency patterns, wherein each of said spatial frequency strength values and / or temporal frequency strength values that is different than other of one or more of said spatial frequency strength values and / or temporal frequency strength values corresponds to and are identified as one of one or more of said unknown devices that produce said intangible attribute inside said real space; determining whether one or more of said temporal frequency values, of one or more of said temporal frequency patterns, is within a predetermined target temporal frequency band and / or one or more spatial frequency values, of one more of said spatial frequency patterns, is within a predetermined target spatial frequency band, wherein each of said temporal frequency patterns having one more of said temporal frequency values that is within said predetermined target temporal frequency band and / or each of said spatial frequency patterns having one or more of said temporal frequency values that is within said target spatial frequency band corresponds to and is identified as one of one or more of said unknown devices that produce said intangible attribute inside said real space;determining whether one or more of said temporal frequency values, of one or more of said temporal frequency patterns matches a target temporal frequency value and / or one or more of said spatial frequency patterns, matches a target spatial frequency value, wherein each of said temporal frequency patterns having one more of said temporal frequency values that matches said target temporal frequency value and / or each of said spatial frequency patterns having one or more of said spatial frequency values that matches said target spatial frequency value corresponds to and is identified as one of one or more of said unknown devices that produce said intangible attribute inside said real space; and comparing, one or more of said temporal frequency strength values, of one or more of said temporal frequency patterns, to a predetermined temporal frequency strength threshold value and / or one or more of said spatial frequency strength values, of one or more spatial frequency patterns, to a predetermined spatial frequency strength threshold value, wherein each of one or more of said temporal frequency patterns having one or more of said temporal frequency strength values that is greater than or equal to said predetermined temporal frequency strength threshold value and / or each of one more of said spatial frequency patterns having one or more of said spatial frequency strength values that this is greater than or equal to said predetermined spatial frequency strength threshold value corresponds to and is identified as one of one or more of said unknown devices that produce said intangible attribute inside said real space.

18. The method for determining location and / or direction of location of one or more unknown devices of claim 16, wherein said identifying includes matching, within a predefined tolerance, one or more of said spatial frequency patterns and / or one or more of said temporal frequency patterns with a spatial frequency reference pattern generated by a particular type of unknown device and / or a temporal frequency reference pattern generated by said particular type of said unknown device.*** SYSTEM CLAIM***19. A system for determining location and / or direction of location of one or more sources of intangible attributes, said system comprising: one or more vehicles, each of which is equipped with or communicatively coupled to one or more non-imaging sensors for measuring, at different instances in time and at one or more three-dimensional coordinates that define a region or a location in a real space, one or moreproperties of one or more different types of said intangible attributes present in said real space to produce an intangible attribute dataset and does not include imaging data of said real space; an imaging system for obtaining an image value dataset that includes said imaging data of said real space and conveys image-related information of said real space and includes one or more of said three-dimensional coordinates, which define a location or a region in said real space of said image-related information; and a processor communicatively coupled to said non-imaging sensor and said imaging system, and programmed for carrying out the following instructions: obtaining, for said real space, said intangible attribute dataset, wherein said intangible attribute dataset includes different instances in time of measurement, one or more said three-dimensional coordinates of the measurement, one or more of said properties measured, and does not include imaging data of said real space; modifying said intangible attribute dataset to produce a source identifying dataset, which includes multiple types of information chosen from a group comprising information regarding said different instances in time of measurement, information regarding one or more of said three-dimensional coordinates of measurement, and information regarding one or more of said properties measured, and does not include imaging data of said real space; identifying, based on said source identifying dataset and without using said imaging data, one or more of said different types of said intangible attribute sources that produce one or more different types of said intangible attributes present inside said real space; determining, based on source identifying dataset and without using said imaging data, one or more locations and / or directions of locations of said one or more of said type of said intangible attribute sources and / or properties of one or more of said types of said intangible attributes; and displaying and / or causing to display, in a virtual space defined inside a user interface of a user device that is communicatively coupled to one or more of said vehicles, one or more said locations and / or said directions of said locations of one or more of said different types of said intangible attribute sources and properties of one or more of said types of said intangible attributes.

20. The system for determining location and / or direction of location of one or more sources of said intangible attributes of claim 19, wherein said imaging system includes one or more imaging sensors mounted on one or more of said vehicles.

Images