Novel systems and methods for visualizing multiple senor signals in a virtual environment

Abstract

Systems and methods for rendering a virtual three-dimensional space scene that includes location of one or more sources of intangible attributes are described. An exemplar method includes: (i) obtaining a first type of dataset; (ii) obtaining a second type of dataset and an offset distance; (iii) performing a fast Fourier transformation the second type of dataset to produce a frequency domain dataset; (iv) identifying one or more of the intangible attribute signals in a real space; and (v) spatializing the first type of dataset and offset to create a first type of spatialized dataset and spatialized offset; (vi) spatializing the second type of dataset to create a second type of spatialized dataset; (vii) aligning, using the spatialized offset, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and (viii) rendering the enhanced three-dimensional spatialized environment on a display component.

Description

NOVEL SYSTEMS AND METHODS FOR VISUALIZING MULTIPLE SENOR SIGNALS IN A VIRTUALENVIRONMENTRELATED APPLICATION

[0001] This application claims priority to provisional application number 63 / 619,321, filed on January 10, 2024, which is incorporated herein by reference for all purposes.FIELD

[0002] The present arrangements and teachings relate generally to novel systems and methods for collecting, locating and visualizing sensor signal measurements in an extended reality, which includes and spans from virtual reality to augmented reality. More particularly, the present arrangements and teachings relate to novel systems and methods, using among other techniques, complex data spatialization, for collecting, locating and visualizing sensor signal measurements in extended reality not visible to the human eye.BACKGROUND

[0003] Virtual reality (“VR”) and augmented reality (“AR”) conventionally use a hand-held controller to effectuate the full features of the technology. Further, each of VR and AR spans a range of reality, which is commonly called “Extended Reality” (“XR”). By way of example, user, using a display interface, will see the XR or virtual world, which broadly describes a world that encompasses both VR, AR, and all the intermediate realities.

[0004] Unfortunately, an XR system, which includes a headset, is typically limited and unable to process large amounts of complex data to convey information about the actual world (which is different from the virtual world) and therefore is not able to effectively inform about the actual world. As a result, applications of the XR system are limited and do not always lend themselves to conveying meaningful information. Moreover, given this limited bandwidth for data processing, it is difficult to design applications, which deploy XR systems, for commercially viable applications.

[0005] What is, therefore, needed are systems and methods that use XR systems and methods and that process large amounts of complex data to inform on real space, without suffering from the drawbacks encountered when using systems and methods involving the current XR technology.SUMMARY

[0006] The present systems and methods described herein relate to processing large amounts of complex data to inform, in a virtual space, on information present in, but not visible to a human eye, in a real space. Moreover, the present systems and methods deploy XR systems to display, in a two-dimensional or three-dimensional virtual space, locations and / or directions of locations of concealed, hidden, or unknown intangible attribute sources, e.g., Wi-Fi™ routers and / or tracking devices that generate a signal not visible to the human eye, using extended reality. Moreover, the present systems and methods deploy XR systems to also determine, in a virtual space, whether an intangible attribute source is properly operating.

[0007] In one embodiment, the present teachings offer methods for rendering two different types of datasets. One such exemplar method includes obtaining a first type of dataset, using a first sensor disposed on an AR / VR headset and that measures a first attribute at one or more three- dimensional coordinates defining a region or location in real space. The first type of dataset includes a first attribute value and an associated the three-dimensional coordinates where the first attribute value is obtained.

[0008] The exemplar method for rendering two different types of datasets includes another obtaining element. The obtaining element includes obtaining a second type of dataset, using a second sensor that couples to the AR / VR headset that operates within a range of frequency that includes one or more frequency blocks. The second sensor is any sensor providing an output that is capable of spectral analysis. The second sensor measures for at least one of the frequency blocks, intensity values of one or more different intangible attributes at one or more of the three- dimensional coordinates in the real space. Moreover, a point of origin of the second sensor is at an offset distance relative to the point of origin of the first sensor

[0009] The exemplar method for rendering two different types of datasets then caries out a performing element that includes performing, for the frequency block, a fast Fouriertransformation on the intensity values to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values.

[0010] Next, the exemplar method proceeds to identifying, based on one or more the frequency peak values and without using the first type of dataset, one or more intangible attribute signals inside the real space.

[0011] Next the exemplar method proceeds to two spatializing elements. The first spatializing element includes spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance

[0012] Another spatializing element includes spatializing, using the plurality of the three- dimensional coordinates, the second type of dataset to create a second type of spatialized dataset.

[0013] Following the two spatializing elements, aligning element include aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment.

[0014] A rendering element includes rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment. One or more of the intangible attribute signals, associated with one or more of the frequency peak values, are visually represented in the enhanced three-dimensional spatialized environment.

[0015] In another embodiment, the present teachings offer methods for rendering gas leakage. One such exemplar method includes obtaining a first type of dataset, using a first sensor disposed on an AR / VR headset and that measures a first attribute at one or more three-dimensional coordinates defining a region or location in real space. The first type of dataset includes a first attribute value and an associated the three-dimensional coordinates where the first attribute value is obtained.

[0016] The exemplar method for rendering gas leakage includes another obtaining element. The obtaining element includes obtaining a second type of dataset, using a second sensor that couples to the AR / VR headset that operates within a range of frequency that includes one or more frequency blocks. The second sensor is any sensor providing an output that is capable of spectral analysis. The second sensor measures for at least one of the frequency blocks, intensity values of one or more different intangible attributes at one or more of the three-dimensional coordinates inthe real space. Moreover, a point of origin of the second sensor is at an offset distance relative to the point of origin of the first sensor

[0017] The exemplar method for rendering gas leakage then caries out a performing element that includes performing, for the frequency block, a fast Fourier transformation on the intensity values to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values.

[0018] Next, the exemplar method proceeds to identifying, based on one or more the frequency peak values and without using the first type of dataset, presence of one or more gas signals associated with one or more gases inside the real space.

[0019] Next the exemplar method proceeds to two spatializing elements. The first spatializing element includes spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance

[0020] Another spatializing element includes spatializing, using the plurality of the three- dimensional coordinates, the second type of dataset to create a second type of spatialized dataset.

[0021] Following the two spatializing elements, aligning element include aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment.

[0022] A rendering element includes rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment and indicating, in the enhanced three-dimensional spatialized environment, presence and concentration of one or more of the gases.

[0023] In one embodiment, the present arrangements relate to an extended reality systems for rendering a user interface. One exemplar extended reality system includes a first sensor disposed on an AR / VR headset. The first sensor measures, at one or more three-dimensional coordinates that define a region or location in real space, one or more first attribute values to produce a first type of dataset. The first type of dataset includes a first attribute value and an associated the three-dimensional coordinates where the first attribute value is obtained.

[0024] The exemplar extended reality system includes a second sensor, coupled to the AR / VR headset for measuring, that operates within a range of frequency that includes one or more frequency blocks. The second sensor measures, for at least one of the frequency block, intensityvalues of one or more different intangible attributes at one or more of the three-dimensional coordinates in the real space. Moreover, a point of origin of the second sensor is at an offset distance relative to the point of origin of the first sensor.

[0025] The exemplar extended reality system includes a display component and a processor that is communicatively coupled to the first sensor, the second sensor, and the display component. The display component displays rendered information.

[0026] The processor is operative to perform the following instructions: (i) performing, for the frequency block, a fast Fourier transformation on the intensity values to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values; (ii) identifying, based on one or more the frequency peak values and without using the first type of dataset, one or more intangible attribute signals inside the real space; (iii) spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance; (iv) spatializing, using the plurality of the three-dimensional coordinates, the second type of dataset to create a second type of spatialized dataset; (v) aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and (vi) rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment wherein one or more intangible attribute signals, associated with one or more of the frequency peak values, are visually represented in the enhanced three-dimensional spatialized environment.

[0027] In another embodiment, the present arrangements relate to an extended reality systems for rendering gas leakage. One exemplar extended reality system includes a first sensor disposed on an AR / VR headset. The first sensor measures, at one or more three-dimensional coordinates that define a region or location in real space, one or more first attribute values to produce a first type of dataset. The first type of dataset includes a first attribute value and an associated the three-dimensional coordinates where the first attribute value is obtained.

[0028] The exemplar extended reality system includes a second sensor, coupled to the AR / VR headset for measuring, that operates within a range of frequency that includes one or more frequency blocks. The second sensor measures, for at least one of the frequency block, intensity values of one or more different intangible attributes at one or more of the three-dimensionalcoordinates in the real space. Moreover, a point of origin of the second sensor is at an offset distance relative to the point of origin of the first sensor.

[0029] The exemplar extended reality system includes a display component and a processor that is communicatively coupled to the first sensor, the second sensor, and the display component. The display component displays rendered information.

[0030] The processor is operative to perform the following instructions: (i) performing, for the frequency block, a fast Fourier transformation on the intensity values to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values; (ii) identifying, based on one or more the frequency peak values and without using the first type of dataset, presence of one or more gas signals associated with one or more gases inside the real space; (iii) spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance; (iv) spatializing, using the plurality of the three-dimensional coordinates, the second type of dataset to create a second type of spatialized dataset; (v) aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and (vi) rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment and indicating, in the enhanced three-dimensional spatialized environment, presence and concentration of one or more of the gases.

[0031] The construction and method of operation of the invention, however, together with additional objects and advantages thereof, will be best understood from the following descriptions of specific embodiments when read in connection with the accompanying figures.BRIEF DESCRIPTION

[0032] Figure 1 shows a block diagram of an XR system, according to one embodiment of the present arrangements and that includes, among other things, an eyewear and a processor to create XR.

[0033] Figure 2 shows an eyewear, according to one embodiment of the present arrangements and that includes imaging devices and displays to view XR renderings of the present teachings.

[0034] Figure 3 shows a second sensor, according to one embodiment of the present arrangements and that is attached to the eyewear of Figure 2.

[0035] Figure 4 shows a block diagram of a processor-based eyewear, according to one embodiment of the present arrangements that integrates a non-imaging sensor, a processor, and an optical sensor, which includes the imaging devices of Figure 2, such that these components function in a cooperative manner to perform certain methods of the present teachings described herein.

[0036] Figure 5A shows a block diagram of the various software modules and a rendering engine, according to one embodiment of the present arrangements, present inside a processor, e.g., inside the processor-based eyewear of Figure 3 or the server of Figure 1.

[0037] Figure 5B shows a block diagram of the various software modules and a rendering engine, according to an alternative embodiment of the present arrangements, present inside a processor, e.g., inside the processor-based eyewear of Figure 3 or the server of Figure 1.

[0038] Figure 6 shows an XR system, according to one embodiment of the present arrangements, that has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0039] Figure 7 shows an XR system, according to another embodiment of the present arrangements and that includes an eyewear communicatively coupled to a smartphone, which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0040] Figure 8 shows an XR system, according to yet another embodiment of the present arrangements and that includes an eyewear communicatively coupled to a network (e.g., the Internet), which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0041] Figure 9 shows an XR system, according to yet another embodiment of the present arrangements and that includes an eyewear communicatively coupled to a personal computer, which has stored therein programmable instructions for carrying out certain methods of the present teachings described herein.

[0042] Figure 10 shows an XR system, according to yet another embodiment of the present arrangements, and that has stored therein programmable instructions for carrying out, in the absence of an eyewear, certain methods of the present teachings described herein.

[0043] Figure 10 shows an exemplary visual representation resulting from a cluster analysis, according to one embodiment of the present teachings and that shows different types of intangible attribute signals (e.g., Wi-Fi™ signal generator and / or Bluetooth™ signal generator) present in real space.

[0044] Figure 11 shows a hand-held controller, according to one embodiment of the present teaching that is communicatively coupled to an XR system to control what is visually represented in a virtual space.

[0045] Figure 12 shows a second type of dataset, according to one embodiment of the present teachings, that includes, for multiple frequency blocks, multiple intangible attribute measurements at different three-dimensional coordinates in real space.

[0046] Figure 13 shows a spatially distributed reference table, according to one embodiment of the present teachings, for a characteristic frequency regime F 1 at a particular time, t=T 1.

[0047] Figure 14 shows a temporally distributed reference table, according to one embodiment of the present teachings, at a position in real space to develop a temporally distributed reference frequency pattern for this frequency regime.

[0048] Figure 15 shows a frequency domain dataset, according to one embodiment of the present teachings, that includes one or more frequency peaks value in multiple frequency blocks and one or more three-dimensional coordinates of each frequency peak value.

[0049] Figure 16 shows a rendered three-dimensional spatialized image, according to one embodiment of the present arrangements and that depicts a distribution of signal strength indicators of an intangible attribute signal within a particular real space of interest.

[0050] Figure 17 shows a rendered three-dimensional spatialized image, according to one embodiment of the present teachings, that depicts gas leaking from a pipe and gas concentration levels at different location in particular real space of interest

[0051] Figure 18 shows a process flow diagram for a method, according to one embodiment of the present teachings, for rendering one or more different types of attribute value datasets.

[0052] Figure 19 shows a process flow diagram for a method, according to one embodiment of the present teachings, for rendering gas leakage.DETAILED DESCRIPTION

[0053] In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without limitation to some or all of these specific details. In other instances, well known process elements have not been described in detail in order to not unnecessarily obscure the invention.

[0054] The present arrangements provide systems of many different configurations for rendering extended realities (“XRs”) of the present teachings. In one XR of the present teachings, two or more different types of datasets are rendered. In one such XR, one dataset may include image data of a real space and another dataset may include data conveying a magnitude of an intangible property inside the real space. In another XR of the present teachings, an image dataset, which is an electronic representation of the real space being perceived by the user, is spatialized and rendered. According to the present teachings, image datasets are not the only type of datasets that may be spatialized before rendering. In fact, one or more different types of second type of datasets present in the real space may be spatialized before rendering.

[0055] The present systems and methods described herein may be deployed using processorbased eyewear (e.g., AR / VR headset or eyeglasses). In one embodiment of the present arrangement, however, a smartphone, in the absence of an eyewear, renders one or more different types of datasets. As a result, use of the eyewear is not necessary, but represents a preferred embodiment of the present arrangements.

[0056] In certain embodiments, the processor-based eyewear of the present arrangements will generally comprise a processor, which includes one or more memory devices operable to provide machine-readable instructions to the processors and to store data. In some of these embodiments, the processor-based eyewear is communicatively coupled to external devices that include one or more sensors for collecting persisting data. In other embodiments, all the necessary sensors are integrated into the processor-based eyewear.

[0057] In one preferred embodiment of the present arrangements, the processor-based eyewear may include data acquired from remote servers. The processor may also be coupled to various input / output (“I / O”) devices for receiving input from a user or another system and for providing an output to a user or another system. These I / O devices may include human interaction devicessuch as keyboards, touch screens, displays, and terminals as well as remote connected computer systems, modems, radio transmitters, and handheld personal communication devices such as cellular phones, smartphones, and digital assistants.

[0058] The processors of the present arrangements may also include mass storage devices that are associated with disk drives and flash memory modules as well as connections through I / O devices to servers or remote processors containing additional storage devices and peripherals.

[0059] Certain embodiments may employ multiple servers and data storage devices thus allowing for operation in a cloud or for operations drawing from multiple data sources. The present teachings and arrangements contemplate that the methods disclosed herein will also operate over a network such as the Internet, and may be effectuated using combinations of several processing devices, memories, and I / O. Moreover, any device or system that operates to effectuate one or more elements, according to the present teachings, may be considered a “server,” as this term is used in this specification, if the device or system operates to communicate all or a portion of the programmable instructions to another device, such as an eyewear, or a smartphone.

[0060] In certain aspects of the embodiments that employ multiple devices, i.e., multiple eyewear devices, servers, and data storage devices may operate in a cloud or may operate, in the absence of the Internet, to draw from multiple data sources. In these configurations, multiple devices may collectively operate as part of a peer-to-peer ad-hoc network. These configurations of the present arrangement would implement “edge computing,” which is a distributed computing paradigm that brings computation and data storage closer to the location where it is needed, to improve response times and save bandwidth. In some of these aspects of the present arrangement, the eyewear would communicate with each other without the Internet, where one of them would act as a sort of communication server, but without any other connectivity.

[0061] The processors of the present system may be, partially or entirely, integrated into a wireless device such as a smartphone, an eyewear, a personal digital assistant (PDA), a laptop, a notebook, or a tablet computing device that operates through a wireless network. Alternatively, the entire processing system may be self-contained on a single device in certain embodiments of the present arrangements.

[0062] The processors of the present systems may be coupled to a three-dimensional (“3D”) controller for interactivity. A 3D controller takes human input and provides signals directing theprocessing system to alter a display of information. Conventional 3D controllers may be used to virtually move through an image displayed on a screen. Described below are certain exemplary embodiments of the processor-based eyewear of the present arrangements.

[0063] Figure 1 shows a functional block diagram of an Extended Reality (“XR”) system 100, according to one embodiment of the present arrangements. XR system 100 includes a server 110 that is coupled to one or more databases 112 and a network 114 (e.g, the Internet). Network 114 may include routers, hubs, and other equipment to effectuate communications between all associated devices. A user may choose to access server 110 by a computer 116 that is communicatively coupled to network 114. Computer 116 includes, among other things, a sound capture device such as a microphone (not shown to simplify illustration). Alternatively, the user may access server 110 through network 114 by using a smartphone 118. Smartphone 118 may connect to server 110 through an access point 120 coupled to network 114. Smartphone 118, like computer 116, may include a sound capture device such as a microphone.

[0064] XR system 100 may include one or more eyewear 122 coupled to network 114 directly, through access point 120, or directly to remote processing devices. By way of example, an eyewear 122 or game controller may be coupled to a processing device for getting user input. This coupling may be wireless using technologies such as “Wi-Fi™,” “Bluetooth™,” or Z- Wave, i.e., a wireless technology standard used for exchanging data between fixed and eyewear over short distances, using short- wavelength UHF radio waves in the industrial, scientific, and medical radio bands, from 2.402 Gigahertz to 2.480 Gigahertz, and building personal area networks (PANs).

[0065] XR system 100 may also include one or more second sensors 124 that are not disposed on eyewear 122. One or more second sensors 124 are communicatively coupled to eyewear 122, by a wired connection and / or wireless connection, using access point 120 that is coupled to network 114, or directly to remote processing devices.

[0066] Computer 116, smartphone 118, and eyewear 122 may be referred to as client devices within XR system 100. Conventionally, client device / server processing operates by dividing the processing between two devices such as a server and a smart device, such as a smartphone or other computing device. The workload is divided between the servers and the client devices (e.g., eyewear, smartphones or wearables, such as eyewear), according to a predetermined specification. By way of example, in a “light client” mode of operation, the server does most ofthe data processing and the client device does a minimal amount of processing, often merely displaying and / or rendering the result of processing performed on a server (e.g., Server 110).

[0067] According to the current disclosure, client device / server applications are structured so that the server provides programmable instructions to one or more different types of client devices and the client devices execute those programmable instructions. The interaction between the server and the client device indicates which instructions are transmitted and executed. In addition, the client device may, at times, provide programmable instructions to the server, which in turn executes them. Several forms of programmable instructions are conventionally known including applets and are written in a variety of languages including Java and JavaScript.

[0068] Client device applications implemented on the client device, and / or server applications implemented on the server, provide for software as a service (SaaS) application where the server provides software to the client device on an as needed basis.

[0069] In addition to the transmission of instructions, client device / server applications also include transmission of data between the client device and server. Often this entails data stored on the client device to be transmitted to the server for processing. The resulting data is then transmitted back to the client device for display or further processing.

[0070] The present teachings recognize that client devices (e.g., computer 116, smartphone 118, or eyewear 122) may be communicably coupled to a variety of other devices and systems such that the client receives data directly and operates on that data before transmitting it to other devices or servers. Thus, data to the client device may come from input data from a user, from a memory on the device, from an external memory device coupled to the device, from a radio receiver coupled to the device, or from a transducer or sensor coupled to the device. In the present arrangements, the sensors may be optical sensors and / or non-imaging sensors. The radio may be part of a wireless communications system such as a “Wi-Fi™” or Bluetooth™ receiver. “Wi-Fi™,” as this term is used in this specification, refers to a family of wireless networking technologies, based on the IEEE 802.11 family of standards, which are commonly used for local area networking of devices and Internet access.

[0071] One or more second sensors 124 that are communicatively coupled to any one of the eyewear, another client device (e.g., a smartphone or a personal computer), or a server may be any of a number of devices or instruments, in one embodiment, non-imaging sensors of the present arrangements detect a magnitude of an intangible property in the real space.

[0072] A client-server system, in which a client device and a server are communicatively coupled to achieve a desired result, may rely on “engines” that include processor-readable instructions (or code) to effectuate different elements of a design. Each engine may be responsible for different operations and may reside, in whole or in part, on a client device, a server, or another device. As disclosed herein a rendering engine, a display engine, a data engine, an execution engine, and a user interface (UI) engine may be employed. These engines may seek and gather information about events from remote data sources. Further, these engines may facilitate the rendering, displaying, and / or causing to display of “processed information,” which is generated by a processor.

[0073] The described methods and systems, of the present teachings, may be effectuated using conventional programming tools including database tools for collecting, storing, and searching through structured data. Moreover, web-based or other programming techniques may be employed to collect information and cause to display, display, or render results. Accordingly, software engines may be created to effectuate these methods and techniques, either in whole or part, depending on the desired embodiment described herein.

[0074] Figure 2 shows an exemplary eyewear (e.g., an AR / VR headset) 222 of the present arrangements that fit around a user’s eyes using a strap 220. In one embodiment, eyewear 122 of Figure 1 is eyewear 222, which includes a frame 202 that is equipped with three imaging devices for the user’s right side, i.e., 210, 214, 218, three imaging devices for the user’s left side, i.e., 208, 212, 216, and two displays, i.e., 206 and 204, one for each eye.

[0075] In the arrangement of eyewear 222, left look-side imaging device 216 provides an image of the space that lies to the left side of the user, left look-front imaging device 212 provides an image of the space that lies in front, but to the left, of the user, and left look-down imaging device 208 provides an image of the space that lies below, but to the left, of the user. Similarly, with respect to the right side of the user, right look-side imaging device 218 provides an image of the space that lies to the right side of the user, right look-front imaging device 214 provides an image of the space that lies in front, but to the right, of the user, and right look-down imaging device 210 provides an image of the space that lies below, but to the right, of the user. As a result, these six imaging devices (i.e., cameras) capture an image of the space surrounding the user. Further, eyewear 222 includes a left eye display 204 and a right eye display 206 for rendering and / or displaying information for the user’s left eye and right eye, respectively.

[0076] Although eyewear 222 is not shown (to simplify illustration and facilitate discussion) to include a non-imaging optical sensor, frame 202, in certain embodiments of the present arrangements, incorporates one or more non-imaging sensors to provide non-imaging type of information regarding a real space of interest to the user. In preferred configurations of these embodiments, frame 202 houses a processor, which serves as a processing subsystem for processing both imaging and non-imaging information obtained using above-described optical sensor (including an imaging device) and non-imaging sensor, respectively. In this embodiment, the processor of eyewear 222, not only generates “processed information” from the imaging and non-imaging information, but also effectively renders or displays and / or causes to display the “processed information” on left eye display 204 and right eye display 206.

[0077] Figure 3 shows an eyewear 322, according to one embodiment of the present arrangements, that include one or more second sensors 324 are disposed on a strap 220. Eyewear 322 is substantially similar to eyewear 222 of Figure 2 and one or more sensors are substantially similar to one or more second sensors 124 for Figure 1 and in a preferred embodiment of the present arrangements, are non-imaging sensors. The present teachings recognize that eyewear 322 may be limited in the number and types of non-imaging sensors that are built into eyewear 322. In certain scenarios, it would be advantageous to use one or more second sensors 324, which are not available in eyewear 322, to measure data and render different datasets (which may include one or more properties of one or more different intangible attributes measured by one or more second sensors 324). Thus, the present arrangements and teachings allow for one or more second sensors 324 to be added an XR system (e.g., XR system 100 of Figure 1) depending on a type datasets a user wants to collect for a particular environment. To this end, one or more second sensors 324 may be installed or attached to strap 320 of eyewear 322. Thus, the functionality and utility of the XR system is increased as it allows capture and rendering of data not available to eyewear 322 on its own.

[0078] However, the location of one or more second sensors 324 is not limited to installation on strap 320. One or more second sensors 324 may attach at another location of the user that is using eyewear 322. For example, the user may hold in their hand one or more second sensor 324 while the XR system is engaging in data collection. In another embodiment of the present arrangements, one or more second sensors 324 may be coupled to another object in the realspace, such as remotely operated vehicle, another eyewear 322, or a client device (e.g., a smartphone or a personal computer).

[0079] Figure 4 shows a processor-based eyewear 422, according to one embodiment of the present arrangement and that includes a processing subsystem 413 that is configured to receive and process both imaging information obtained from an optical sensor device 405 and nonimaging device information or, in the alternative, non-imaging information, obtained from a nonimaging sensor, for example second sensor 440 (e.g. non-imaging sensor 324 of Figure 3). Such processing of information allows processing subsystem 413 to produce the “processed information” that is rendered and / or displayed, or caused to display, on an I / O interface 417 (e.g., on a display screen or AR / VR headset). A network (e.g., the Internet), preferably, serves as a communication bus 465 to effectively communicatively couple processing subsystem 413, optical sensor device 405, second sensor 440, and I / O interface 417 so that the requisite information from the sensors is received, processed and then displayed and / or rendered on an I / O interface 417.

[0080] Second sensor 440 is a sensor that is configured to measure a magnitude of one or more intangible properties present within a real space and is distinct and / or located remote from a frame of eyewear 422. Measurements obtained by second sensor 440 may be transmitted to processing subsystem 413 for further processing. Second sensor 440 may be hardware, software, or a combination of software and hardware. By way of example, second sensor 440 may represent, or include, radio frequency sensors (“RF sensors”) configured to receive and detect the radio signals (or RF signals). Examples of RF sensors may include, but are not limited to, RF antennas and RF identity (RFID) readers. The RF sensors may function as transducers to convert electromagnetic (EM) signals into electrical signals for processing by the processor. However, in one implementation of the present arrangements, at least one of second sensor 440 may be implemented as a sensor device including a processor. For example, second sensor may include RF sensors combined with a processor in a single package to receive, detect, and measure EM signals in the radio frequency range.

[0081] In preferred embodiments of the present arrangements, second sensor 440 is at least one sensor chosen from a group including sensor measuring throughput of a connectivity signal, sensor measuring latency of a connectivity signal, sensor measuring interference of a connectivity signal, sensor measuring volatility of a connectivity signal, sensor measuringstability of a connectivity signal, radio frequency (hereinafter also referred to as “RF”) power output sensor, a radio frequency receiver sensor (e.g., a RF receiver module or a RF transceiver module), electric and magnetic fields (“EMF”) sensor, atmospheric pressure sensor, geomagnetic sensor, hall effect sensor, ambient light level sensor, gas levels sensor, smoke sensor, sound pressure sensor, audio harmonics sensor, humidity sensor, carbon dioxide emission sensor, and temperature sensor. In one embodiment of the present arrangements, a sensor measuring a throughput of a connectivity signal measures a bandwidth of the connectivity signal, and sensor measuring volatility of a connectivity signal measures the stability of the connectivity signal.

[0082] Optical sensor device 405 of Figure 4 includes an imaging sensor 415 (e.g., imaging look-down imaging sensor 208 of Figure 2), an inertial measurement unit (“IMU”) 442, and one or more position sensors 435. Processing subsystem 413 includes an image processing engine 419, an application store 421, and a tracking module 423. Optical sensor device 405 may optionally include an optical assembly 430 and / or an electronic display 425.

[0083] In optical sensor device 405, one or more imaging sensors 415 capture data characterizing a scene or a local area of interest, typically the space surrounding a user. In accordance with one present arrangement, imaging sensor 415 includes a traditional image sensor (e.g., camera), such that the signals captured by imaging sensor 415 include only two- dimensional image data (e.g., data having no depth information). In another present arrangement, imaging sensor 415 includes a projector device that allows imaging sensor 415 to operate as a depth imaging system that computes depth information for a scene using collected data (e.g., based on captured light according to one or more computer-vision schemes or algorithms, by processing a portion of a structured light pattern, by time-of-flight (“ToF”) imaging, or by Simultaneous Localization And Mapping (“SLAM”)). In an alternative present arrangement, imaging sensor 415 transmits corresponding data to another device, such as the processing subsystem 413, which determines or generates the depth information using the data from the imaging sensor 415.

[0084] Imaging sensor 415, in yet another present arrangement, is a light detection and ranging (“LiDAR”) sensor that transmits a pulse of infrared light and receives data that is refracted back to the LiDAR sensor to create a three-dimensional map of a scene or local area of interest. In yet another embodiment of the present arrangements, imaging sensor 415 is a radio detection andranging (“RADAR”) sensor that transmits radio waves and receives data reflected back to the RADAR sensor to create a three-dimensional map of a scene or local area of interest.

[0085] In yet another present arrangement, imaging sensor 415 is a hyperspectral imaging device that represents a scene as multiple spectra of light. In this arrangement, different features or objects within a scene are visualized using light of specific wavelengths, and, therefore, may be better understood, analyzed, and / or visually or quantitatively described.

[0086] In preferred embodiments of the present arrangements, optional electronic display 425 displays two-dimensional or three-dimensional images based upon the type of data received from processing subsystem 413. In certain of these arrangements, optional optical assembly 430 may magnify image light received from electronic display 425, correct optical errors associated with the image light, and / or present the corrected image light to a user of optical sensor device 405.

[0087] In one embodiment of the present arrangements, IMU 442 represents an electronic device that generates fast calibration data based on measurement signals received from one or more of the position sensors 435. To this end, one or more position sensors 435 are configured to generate one or more measurement signals in response to motion of eyewear 422. If one or more of position sensors 435 provide information on three-dimensional locations where measurements are obtained, then in this specification, they are sometimes (e.g., in connection with Figures 10, 11, and 13) referred to as a “ground position component.”

[0088] Examples of different types of position sensors 435 include accelerometers, gyroscopes, magnetometers, another suitable type of sensor that detects motion, or a type of sensor used for error correction of IMU 442. Position sensors 435 may be located external to IMU 442, internal to the IMU 442, or some portions may be located internal to, and other portions may be located external to IMU 442. Regardless of their location, position sensors 435 may detect one or more reference points, which are used for tracking a position of eyewear 422 in a local area by using, for example, tracking module 423 of processing subsystem 413.

[0089] In processor subsystem 413, image processing engine 419 may generate, based on information received from optical sensor device 405 or from components thereof (e.g., imaging sensor 415), a three-dimensional depth mapping or multiple three-dimensional depth mappings of the space (e.g., the "scene" or the "local area" of interest) surrounding a portion or all of optical sensor device 405. In certain embodiments, image processing engine 419 of the present arrangements may generate depth information for the three-dimensional mapping of the scenebased on two-dimensional information or three-dimensional information received from imaging sensor 415 that is relevant for techniques used in computing depth maps. The depth maps may include depth dimension values for each of the pixels in the depth map, which may represent multiple different portions of a scene.

[0090] Continuing with processor subsystem 413, application store 421 may store one or more software applications or programmable instruction sets for execution by processing subsystem 413 or by the optical sensor device 405. A software application may, in some examples, represent a group of programmable instructions that, when executed by a processor (e.g., processing subsystem 413 of Figure 3 or server 110 of Figure 1), generate or render content for presentation to the user. Content generated or rendered by a software application may be generated or rendered in response to inputs received from the user via movement of the optical sensor device 405 or I / O interface 417.

[0091] Examples of software applications, stored on application store 421, include gaming applications, conferencing applications, video playback applications, or programmable instructions for performing the various methods described herein. In preferred embodiments of the present arrangements, application store 421 may be a non-transitory memory store that also stores data obtained from second sensor 440, imaging sensor 415, or from other sources included in optical sensor device 405 or received from processing subsystem 413.

[0092] Tracking module 423 may calibrate eyewear 422 using one or more calibration parameters and may adjust the calibration parameters to reduce error in determination of the position of optical sensor device 405 or of I / O interface 417. Additionally, tracking module 423 may track movements of second sensor 440, optical sensor device 405, or of I / O interface 417 using information from imaging sensor 415, to one or more position sensors 435, IMU 442, or some combination thereof. I / O interface 417 may represent a component that allows a user to send action requests and receive responses from processing subsystem 413. In some embodiments of the present arrangements, an external controller may send such action requests and receive such responses via I / O interface 417. An action request may, in some examples, represent a request to perform a particular action. By way of example, an action request may be an instruction to start or end capture of image or video data or an instruction to perform a particular action within a software application. I / O interface 417 may include one or more input devices. Exemplary input devices include keyboard, mouse, hand-held controller, or any othersuitable device for receiving action requests and communicating the action requests to processing subsystem 413.

[0093] Further, I / O interface 417 may permit eyewear 422 to interact, via a wired or wireless channel, with external devices and / or system accessories, such as additional standalone-sensor systems or hand-held controllers. In preferred embodiments of the present arrangements, optical sensor device 405, processing subsystem 413, and / or I / O interface 417 may be integrated into a single housing or body. Other embodiments may include a distributed configuration of eyewear 422, in which optical sensor device 405 may be in a separate housing or enclosure, but still coupled to processing subsystem 413 by a wired or wireless communication channel. By way of example, optical sensor device 405 may be coupled to processing subsystem 413 that resides inside or is provided by an external gaming console or an external computer, such as a desktop or laptop computer. Processing subsystem 413 may also be a specialized hardware component designed to cooperate specifically with optical sensor device 405 to perform various operations described herein.

[0094] Eyewear 422 may use, among other things, a matrix of variable-phase optical elements (e.g., diffractive optical elements (DOEs)) to introduce phase delays into a wavefront of light received through a lens, thereby enhancing the performance of optical sensor device 405, or specifically one or more of imaging sensors 415 that capture aspects of a scene. These enhancements may be a function of how light passes through the variable-phase optical elements and, in some present arrangements, may also be a function of shifting the matrix or another optical component (e.g., a sensor or a lens) of eyewear 422. By way of example, the phase delays introduced by the matrix of variable-phase optical elements may enable eyewear 422 to capture at least two different perspectives of a scene, and the different embodiments of XR systems described herein may use these different perspectives to provide or increase resolution (e.g., in an angular, depth, and / or spectral dimension) of output images or frames obtained from optical sensor device 405.

[0095] Different embodiments of an optical sensor device (e.g., optical sensor device 405 of Figure 4) disclosed herein may use variable-phase optical elements to capture different perspectives of a scene in a variety of different manners and for numerous different purposes. By way of example, a DOE may be configured to, while in an initial position, disperse light from a scene as an interference pattern on the optical sensor device, which may capture the interferencepattern as a first perspective of the scene. The DOE may be shifted laterally to a subsequent position such that the resulting interference pattern represents another perspective of the scene, which may also be captured by the optical sensor device. These two perspectives may be processed to increase angular resolution (e.g., via oversampling) or to provide depth sensing (e.g., via triangulation and / or phase discrimination).

[0096] By way of example, depth values of a scene may be obtained using triangulation between two perspectives, by using a DOE to provide the two perspectives to a single optical sensor device. As another example, each element within a layer or matrix of variable-phase optical elements may be configured to deterministically phase-shift and focus light onto particular pixels (or sets of pixels) of the optical sensor device. These phase-shifted wavefronts, which may represent different perspectives of a scene, may be captured, mixed, and compared against a reference signal to detect depth within a scene.

[0097] Embodiments of the present arrangements described herein may also be implemented within various types of systems (e.g., traditional CMOS sensor systems, time-of-flight (ToF) systems, or hyperspectral imaging systems) having diverse configurations (e.g., configurations with static or movable optical components). As an example of an implementation with movable optical components (e.g., when a user chooses to move an optical sensor), the optical sensor device may include a matrix of variable-phase optical elements positioned over individual pixels or voxels of an imaging device and an actuator configured to move a component of the optical sensor device (e.g., the matrix, a sensor or a lens) to obtain two different images representing two different instantaneous fields of view (iFOVs) per pixel. The system may then analyze these images to obtain or deduce additional spatial information for the imaged scene. In some examples with a ToF sensor, a scene may be captured in greater spatial resolution by using a conventional large pixel ToF sensor system and translating the component to oversample the portion of the image plane or scene. In examples with a non-ToF sensor (e.g., a traditional CMOS sensor), the system may perform a triangulation operation and / or a phase-discrimination operation on the different iFOVs to calculate a depth map of the scene. The system may also, for both non-ToF and ToF sensors, interpolate between the phase-shifted iFOVs to improve angular resolution of images captured by the sensors.

[0098] The oversampling process may also be used to increase spatial resolution in various hyperspectral imaging systems (e.g., snapshot hyperspectral imaging systems). Traditionalhyperspectral imaging may use hyperspectral filters (e.g., tiled filters or mosaic filters) disposed directly on an imaging device to sample broadband light in the spectral domain, which may increase spectral resolution at the expense of spatial resolution. In contrast, the proposed hyperspectral imaging system may decouple the hyperspectral filters from the imaging device and position the variable-phase optical elements between the filters and the imaging device to facilitate spatial oversampling and improved spatial resolution. For example, a scene may be captured in a hyperspectral image in greater spatial resolution by translating the variable-phase optical elements to oversample portions of the image plane or scene through the individual windows of the hyperspectral filter.

[0099] In one embodiment of the present optical sensor device, each optical component is fixed in a single position and / or movable among two or more positions in a plane perpendicular to the optical axis. For example, a system with fixed optical components may introduce two or more different phase shifts in an incident wavefront. These phase-shifted signals may then be mixed and compared with a reference signal. As another example, a global shutter system may include optical elements that create two phase-shifted optical paths that are captured and stored by the imaging device while the optical elements are in a first position. The system may then shift the optical elements to a second position to create two additional phase-shifted optical paths, which may also be captured by the imaging device. As a result, the imaging device may simultaneously provide four phase-shifted signals to an electrical quadrature demodulation component, where they may be mixed and compared to a reference signal to create a depth map of a scene.

[0100] Figure 5A shows a block diagram of the various software modules and an engine, each of which includes programmable instructions to carry out one or more elements, involved in the implementation of different methods according to the present teachings. A processing subsystem 513 present in a client device (e.g., processing subsystem 413 of an eyewear 422 of Figure 4) or in a server (e.g., server 110 of Figure 1) includes an image spatializing module 576, an attribute spatializing module 578, an aligning module 580 and a rendering engine 582.

[0101] Image spatializing module 576 includes, among other things, programmable instructions for spatializing a first type of pixel or voxel data to create a first type of three-dimensional spatialized data. Attribute spatializing module 578 includes, among other things, programmable instructions for spatializing a second type of three-dimensional pixel or voxel data to create a second type of three-dimensional spatialized data, which is of different type than the first type ofthree-dimensional spatialized data. Aligning module 580 includes, among other things, programmable instructions for aligning the first type of three-dimensional spatialized data with the second type of three-dimensional spatialized data to create an enhanced three-dimensional spatialized environment. Rendering engine 582 includes, among other things, programmable instructions for rendering, on a display component, the “processed information,” i.e., an enhanced three-dimensional spatialized environment. Although rendering engine 582 is described as a preferred embodiment, the present arrangements contemplate use of other types of engines, such as a display engine, a data engine, an execution engine, or user interface (UI) engine to achieve different results contemplated by the present teachings.

[0102] It is not necessary that each of the different modules and engines presented in Figure 5A be disposed on a processor subsystem. According to the present arrangements, these modules and engines may include a distributed configuration, in which certain modules may reside on an optical sensor device (e.g., optical sensor device 405 of Figure 4) that is coupled to a processing subsystem (e.g., processing subsystem 413, or one that resides on a smartphone or a personal computer) by a wired or wireless communication channel. In this configuration, the processing subsystem is, preferably, a specialized hardware component designed to cooperate specifically with an optical sensor and / or a non-imaging sensor to perform various operations described herein.

[0103] Figure 5B shows one exemplary distributed configuration of the modules and engine shown in Figure 5A. According to Figure 5B, a processor subsystem 513' includes aligning module 580' and rendering engine 582', each of which is substantially similar to their counterparts shown in Figure 5A, i.e., aligning module 580 and rendering engine 582. Processing subsystem 513' of Figure 5B is communicatively coupled via a network ( .g., The Internet) 565 to an optical sensor device 505 and a non-imaging sensor device 540 (e.g., second sensor 440 of Figure 4). Further, optical sensor device 505 is similar to optical sensor device 405 of Figure 4, except that optical sensor device 505 includes an image spatializing module 576'. Similarly, nonimaging sensor device 540 is similar to second sensor 440 of Figure 4, except that non-imaging sensor device 540 includes an attribute spatializing module 578’. The programmable instructions contained inside image spatializing module 576' and attribute spatializing module 578’ of Figure 5B are substantially similar to those found in image spatializing module 576 and attribute spatializing module 578 of Figure 5 A, respectively. Tn this distributed configuration, the imagespatializing module in the optical sensor device and the modules and rendering engine in the processor subsystem cooperate to carry out the present methods described herein. Attribute spatializing module 578’, in another embodiment of the present arrangements, is included in processor subsystem 513’, and not attribute spatializing module 578’.

[0104] Figure 6 shows an XR system 600, according to one embodiment of the present arrangements that includes eyewear 622 and one or more sensors 624. In this embodiment, eyewear 622 has stored therein programmable instructions 200 for carrying out certain methods of the present teachings described herein. In one embodiment of the present arrangements, eyewear 622 is substantially similar to at least one of eyewear 122 of Figure 1, eyewear 222 of Figure 2, or eyewear 322 of Figure 3 and one or more sensors 624 is substantially similar to one or more second sensors 124 of Figure 1, one or more sensors 324 of Figure 3, and second sensor 440 of Figure 4. Moreover, one or more sensors 624 are commutatively coupled, in a wired or wireless manner, to eyewear 622 and data obtained by one or more sensors 624 are used by programmable instructions 200. In one preferred embodiment of the present arrangements, programmable instructions 200 include an image spatializing module 576, an attribute spatializing module 578, an aligning module 580, and a rendering engine 582 of Figure 5A.

[0105] Figure 7 shows an XR system 700, according to one embodiment of the present arrangements, that includes eyewear 722, one or more sensors 725, and a smartphone 716. In this embodiment, eyewear 622, which is substantially similar to at least one of eyewear 122 of Figure 1, eyewear 222 of Figure 2, or eyewear 322 of Figure 3, and one or more sensors 725, which is substantially similar to one or more second sensors 124 of Figure 1, one or more sensors 324 of Figure 3, and second sensor 440, is communicatively coupled to a smartphone 716. In one aspect of this embodiment, smartphone 716 is substantially similar to smartphone 118 of Figure 1. Further, in XR system 700 of Figure 7, smartphone 716 has programmable instructions 200 stored therein for carrying out certain methods of the present teachings described herein. In an alternative embodiment of the present arrangements, modules and engines that comprise programmable instructions 200 are distributed between smartphone 716 and eyewear 722. In one implementation of this alternative embodiment, the distributed configuration of software modules is implemented between a processor subsystem that resides inside smartphone 716 and eyewear 722. The present arrangements recognize that programmable instructions, in a distributed configuration, are not limited to client devices.

[0106] To this end, Figure 8 shows an XR system 800, according to an alternative embodiment of the present arrangements. In this embodiment, eyewear 822 and one or more sensors 824 are communicatively coupled to a network (e.g., the Internet) 714, which has stored therein either all of or at least some of the modules or engines that comprise programmable instructions 200. Eyewear 822 is substantially similar to at least one of eyewear 122 of Figure 1, eyewear 222 of Figure 2 or eyewear 322 of Figure 3, and network 814 is substantially similar to at least one of network 114 of Figure 1, communication bus 465 of Figure 4 or network 565 of Figure 5B. One or more sensors 824 is substantially similar to one or more second sensors 124 of Figure 1, one or more sensors 324 of Figure 3, and second sensor 440 of Figure 4.

[0107] Figure 9 shows an XR system 900, according to yet another embodiment of the present arrangements, that includes eyewear 922, one or more sensors 924, and a personal computer 918. In this embodiment, eyewear 922 and one or more sensors are communicatively coupled to a personal computer 916, which may function as a client device and / or a server. In one embodiment, a processing subsystem, which resides inside personal computer 916, has stored therein programmable instructions 200 for carrying out certain methods of the present teachings described herein. In an alternative implementation of the XR system shown in Figure 9, the modules and engine that includes programmable instructions 200 are distributed between personal computer 916 and eyewear 922.

[0108] Figure 10 shows an XR system 1000, according to another embodiment of the present arrangements and that is a smartphone 1018, which has stored therein programmable instructions 200. In this embodiment, smartphone 1018 includes a display, upon which the “processed information” resulting from analysis of different types of datasets is rendered. As a result, XR system 1000 of Figure 9 does not require an eyewear to perform certain methods of the present teachings described herein.

[0109] Figure 11 shows a hand-held controller 1150, according to one embodiment of the present teachings. Hand-held controller 1150 interacts with an VO interface (e.g., I / O interface 417 of eyewear 422 of Figure 4). Hand-held controller 1150 may send action requests (e.g., a selection signal) via the VO interface. An action request may, for example, represent a request to perform a particular action within the software application or move a user within an enhanced three-dimensional spatialized environment. Hand-held controller 1150 includes one or morecontrollers of a first type 1152, one or more controllers of a second type 1154, and one or more light emitting diodes (“LEDs”) 1155.

[0110] In one embodiment of the present arrangements, one or more controllers of a first type 1152 are one or more buttons, which may be engaged to activate and / or deactivate a function, and one or more controllers of a second type 1154 are sliders, which may be moved along a track to increase and / or decrease another function. By way of example, each button (i.e., controller of the first type 1152) is associated with an intangible attribute signal that may be visually represented in an enhanced three-dimensional spatialized environment. A user, using hand-held controller 1150, may add or remove an intangible attribute signal from the enhanced three- dimensional spatialized environment by engaging the button associated with that an intangible attribute signal.

[0111] In another embodiment of the present arrangements, a user, using hand-held controller 1150, may select an intangible attribute signal that is visually represented in the enhanced three- dimensional spatialized environment. To select the intangible attribute signal, the user engages the button associated with that intangible attribute signal. The user may also engage controller of a second type 1154 (e.g., slider) that is associated with the selected intangible attribute signal to increase or decrease the opacity of the selected intangible attribute signal that is visually represented in the enhanced three-dimensional spatialized environment. In another embodiment of the present arrangements, engaging controller of a second type 1154 (e.g., slider) that is associated with the selected intangible attribute signal shows a numerical value of an intensity value of the selected intangible attribute signal.

[0112] One or more light emitting diodes (“LEDs”) 1155 may be used for determining a location of hand-held controller 1150 in real space. A tracking module e.g., tracking module 423 of Figure 4) may use one or more LEDs 1155 to track movements of hand-held controller 1150 using information from an imaging device (e.g. imaging sensor 415 of Figure 4), one or more position sensors (e.g., one or more position sensors 435 of Figure 4, an IMU (e.g., IMU 442 of Figure 4), or some combination thereof. By way example, “inside-out” tracking may be effectuated by the tracking module, in combination with one or more imaging devices and one or more LEDs 1155 to determine the position of hand-held controller 1150 within the real space. Additional imaging devices, such as infrared sensors, may be included to improve tracking of hand-held controller 1 150.

[0113] Figure 12 shows an exemplar second type of dataset 1200, according to one embodiment of the present arrangements, resulting from a second sensor (e.g., one or more second sensors 124 of Figure 1, one or more second sensors 324 of Figure 3, second sensor 440 of Figure 4, non-imaging sensor device 540 of Figure 5B) that is coupled to an AR / VR headset. Second type of dataset 1200 may be obtained during element 1804 of Figure 18. The second sensor operates within a range of frequency that includes one or more frequency blocks (e.g., FBI and FB2) and obtains measurements (e.g., 1202-1, 1202-2, 1202-3) of an intangible attribute at different locations in real space, i.e., different values of X-, Y-, and Z-coordinates that define real space. The second sensor measures, for at least one of the frequency blocks, property values (e.g., Il, 12, 13, 14, and 15) at one or more three-dimensional coordinates in the real space. In one embodiment of the present arrangements, the property is intensity or strength of an intangible attribute present in the real space. The dataset is collected over a period of time (i.e., in a time domain) and starting from t=0 to subsequent instances in time, i.e., at t=Tl, T2, and T3.

[0114] The present arrangements and teachings recognize that one or more intangible attribute signals that generate the property (e.g., intensity) measured by the second sensor may vary spatially, i.e., with changing values of X, Y, and Z, and / or that vary temporally, i.e., with changing values of t (i.e., at t=Tl, T2 and T3). As a result, the present teachings may include a processor being configured to produce a second type of dataset in the form of a spatially distributed second type of dataset 1200 and / or a temporally distributed second type of dataset 1200.

[0115] Figure 13 shows an exemplar spatially distributed second type of dataset 1300, according to one embodiment of the present teachings, for a frequency block (e.g., FBI). Spatially distributed second type of dataset 1300, which is an intangible attribute dataset (i.e., a second type of dataset obtained in element 1804 of Figure 18), according to one embodiment of the present teachings, shows that the amplitude of a frequency regime changes with respect to three- dimensional locations in the real space.

[0116] Figure 14 shows an exemplar temporally distributed second type of dataset 1400, according to one embodiment of the present teachings, for a frequency block. At a location in real space, XI, Yl, Zl, the measured property value (e.g., intensity) does not change with respect to time.

[0117] Figure 15 shows an exemplar frequency domain dataset 1500, according to one embodiment of the present arrangements. Frequency domain dataset 1500 results from transforming one or more intensity values of a frequency block, measured at different instances in time, into frequency domain. Transformation of one or more frequency values into the frequency domain, preferably occurs for reach frequency block. Frequency domain dataset 1500 may be the result of carrying out element 1806 of Figure 18. As discussed above, in each frequency block, one or more intensity values for a particular three-dimensional location in real space may be measured a different instance of time to generate a second type of dataset (e.g., second type of dataset 1200 of Figure 12). In one embodiment of the present arrangements, a fast Fourier transform is used to transform the intensity values measured at different instances in time into frequency domain dataset 1500. Thus, frequency domain dataset 1500 may be thought of as a temporal frequency domain dataset, in that temporal measurements are converted into a frequency domain.

[0118] Frequency domain dataset 1500 includes, at one or more three-dimensional locations (e.g., XI, Yl, Zl) within each frequency block (e.g., FBI), one or more frequency peaks values (e.g., Fl) and a corresponding intensity (e.g., II). For each three-dimensional location within a frequency block, one or more frequency peak values may be the same or different. In other words, depending on the three-dimensional location in real space, a particular three-dimensional location may have the same or different frequency peak values as another three-dimensional location in real space. Further, the intensity value associated with each frequency peak value may change depending on its location within the real space.

[0119] Figure 16 shows a rendering of an interior environment 1600, according to one embodiment of the present arrangements on a display 1625. In this rendering of interior environment 1600, a three-dimensional image 1610 is overlaid with signal strength indicators 1612 (“SSI”) of an intangible attribute signal, which are represented as virtual bars (in dashed line), and virtual hand-held controller 1650, which is a virtual representation of a hand-held controller (e.g., hand-held controller 1150 of Figure 11) located in real space.

[0120] SSI 1612, the original data for which is obtained from element 1804 of Figure 18, is integrated into three-dimensional image 1610, the original data for which is obtained from element 1802 of Figure 18. The “processed information,” underlying rendered interior environment 1600, is developed using elements 1810 and 1812 of Figure 18. In the ultimatelyrendered interior environment 1600, the rendered image data, present in a real space, is shown with boundaries of solid lines, and the rendered attribute data, i.e., SSI 1612, is shown with boundaries of dashed lines. As a result, rendered interior environment 1600 clearly conveys distribution information of SSI, at different locations, inside the office space.

[0121] In some embodiments of the present teachings, different colors of SSI 1612 are used to convey intensity of the intangible attribute signal in real space, for example, obtained from different frequencies (i.e., frequency peak values). In one embodiment, the color representation of a virtual bar, which represents a location of an intensity measurement of an intangible attribute, i.e., SSI, may be presented in a color gradient that vary along a wavelength spectrum and, preferably, range from a green color representation to a red color representation. In one preferred embodiment, a green color representation conveys that a measured attribute value of a particular type is less than an attribute threshold value (e.g., SSI) of the particular type, and the red color representation conveys that the attribute value of the particular type is greater than the attribute threshold value of the particular type. The present teachings recognize that this color- coded virtual bar representation easily conveys SSI distribution information and represents one of the many advantages of the present teachings that is not available using conventional graphical techniques. The present teachings also recognize that virtual bars for SSI 1612, not only provide actual measured information, but may also provide an estimated or a predicted value of the magnitude of SSI 1612.

[0122] While interior environment 1600 illustrates a single intangible attribute signal, represented by SSI 1612, the present teaching and arrangements are not so limited. Multiple intangible attribute signals may be rendered in interior environment 1600. By way of example, at each location, two or more SSI 1612, may be rendered where each SSI 1612 is associated with a different intangible attribute signal. To distinguish between intangible attribute signals, where each SSI 1612, associated with a different intangible attribute signal, have associated a unique color, shading, pattern, or virtual object that distinguishes one signal strength indicator from another. As a result, rendered interior environment 1600 clearly conveys distribution information of SSI 1612, at different locations, inside the office space of multiple intangible attribute signals.

[0123] Further, a user may select, using the hand-held controller located in real space, one selected SSI 1612, which may be highlighted and numerical values associated with SSI 1612 are presented on the display to provide more concrete information about the intensity or magnitudeof SSI 1612. Virtual hand-held controller 1650 may visually render, which buttons or controls the user engaged with on the real world hand-held controller.

[0124] Further still, the user may select a particular intangible attribute signal to render on display 1625 by engaging a button (e.g., controller of a first type 1152 of Figure 11) associated with that intangible attribute signal. The user may also engage with a slider (e.g., controller of a second type 1154 of Figure 11) associated with that button to increase or decrease the opacity of SSI 1412 for that particular intangible attribute signal.

[0125] Figure 17 shows a rendering of an area of interest 1700, according to one embodiment of the present arrangements, that shows a gas leak from a pipe 1702, where the intensity (e.g., gas concentration) of the gas leaking from pipe 1502 is visually represented my multiple color and / or shading. In one embodiment of the present arrangements, a first intensity range 1708 is represented by a first color (e.g., red), a second intensity range 1706 is represented by a second color (e.g., yellow), and a third intensity range 1704 is represented by a third color (e.g., green). Second intensity range 1706 is of a lower magnitude than first intensity range 1708 and third intensity range 1704 is lower than first intensity range 1708 and second intensity range 1706. In this visual rendering, a user viewing area of interest 1700, is quickly and easily able to see the location where gas is leaking from pipe 1702.

[0126] In one aspect, the present teachings offer methods for rendering two or more different types of datasets. Figure 18 shows a flowchart of a method 1800, in accordance with certain embodiments of the present teachings, for rendering two different types of datasets. Method 1800, preferably, begins with element 1802, which includes obtaining a first type of dataset using a first sensor disposed on an AR / VR headset. The first sensor measures a first attribute at one or more three-dimensional coordinates defining a region or location in real space. To this end, the first type of dataset includes a first attribute value and an associated three-dimensional coordinate where the first attribute value is obtained.

[0127] Following element 1802, method 1800 of the present teachings, preferably proceeds to an element 1804 which includes obtaining a second type of dataset (e.g., second type of dataset 1200 of Figure 12) using a second sensor that couples to the AR / VR headset. The second sensor operates within a range of frequency that includes one or more frequency blocks and measures for at least one of the frequency blocks, intensity values at one or more of the three-dimensional coordinates in the real space. Moreover, a point of origin of the second sensor is at an offsetdistance relative to the point of origin of the first sensor. The second sensor, in a preferred embodiment of the present teachings, is a software defined radio sensor.

[0128] Following elements 1804, element 1806 includes performing, for the frequency block, a fast Fourier transformation on the intensity values of one or more different intangible attributes to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values. Preferably, a fast Fourier transformation is performed on each frequency block in which the second sensor measured intensity values at one or more of the three-dimensional coordinates in the real space.

[0129] An element 1808 is carried out after element 1806. Element 1808 includes identifying, based on one or more the frequency peak values and without using the first type of dataset, one or more intangible attribute signals inside the real space. Identifying the presence of one or more of the different types of intangible attribute signals, in one embodiment of the present teachings, includes matching at least part of the frequency domain dataset with a known intangible attribute signal.

[0130] Method 1800 may proceed to an element 1810, which includes spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance.

[0131] Following element 1812, an element 1814 includes spatializing, using the plurality three- dimensional coordinates, the second type of dataset to create a second type of spatialized dataset.

[0132] An element 1816 is carried out after element 1810 and element 1812. Element 1814 includes aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment.

[0133] Following element 1814, an element 1816 is performed. Element 1816 includes rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment. One or more intangible attribute signals, associated with one or more of the frequency peak values, are visually represented in the enhanced three- dimensional spatialized environment.

[0134] Returning to element 1802, in one embodiment of the present teachings, the first sensor is an imaging sensor. A ground positioning component (e.g., position sensor 435 of Figure 4) and the imaging sensor provide or facilitate provision of the first type of dataset. The groundpositioning component informs on the position of the measurements taken when obtaining the image value dataset. The ground positioning component, in preferred embodiments of the present arrangements, uses relative positioning techniques, as opposed to making an absolute position measurement, as encountered when using a global positioning system (“GPS”) sensor, to arrive at the three-dimensional coordinates. Further, the three-dimensional coordinates, in these preferred embodiments, are obtained using positioning measurements relative to one or more reference points or coordinates. The ground positioning component may be disposed in eyewear (e.g., eyewear 222 of Figure 2 or eyewear 422 of Figure 4). The systems and method obtaining the position of measurements take from the second sensor, discussed in element 1804, may also be used to obtain the position of measurements taken from an imaging sensor.

[0135] Eyewear (z.e., AR / VR headset) may include components and / or modules for determine the three-dimensional coordinate of each measurement such as an IMU (e.g., IMU 442 of Figure 4) and one or more position sensors (e.g., one or more position sensors 435 of Figure 4), and a tracking module (e.g., tracking module 423 of Figure 4). A global positioning system (“GPS”) receiver and / or a Global Navigation Satellite System (“GNSS”) receiver may also be mounted onto of communicatively coupled to each eyewear to provide position, velocity, and timing information.

[0136] Moreover, one or more position sensors and / or modules may be coupled to the first sensor to provide three-dimensional coordinates of the fist sensor in real space. Known location or position systems such as Differential GPS (“DGPS”), Real Time Kinematic (“RTK-GPS”), Integrated Internal Navigation, Visual Odometry System, Simultaneous Localization and Mapping (“SLAM”) System, Ultra-Wideband (“UWB”) Positioning System, Long-wave Infrared (“LWIR”), Infrared, and Multi-Constellation Global Navigation Satellite System (“GNNS”) system, may be used individually or in combination, to identify three-dimensional coordinate of each non-imaging sensor and / or each imaging sensor for each measurement with a high degree of accuracy.

[0137] In one implementation of the present teachings, the first attribute value, obtained from an imaging sensor, is a three-dimensional pixelated data value or three-dimensional voxelated data value. Where three-dimensional pixelated data values are obtained as the first attribute value, the three-dimensional coordinates are the three-dimensional coordinates associated with threedimensional pixelated data values. Similarly, where three-dimensional voxelated data values areobtained, as the first attribute value, the three-dimensional coordinates are the three-dimensional coordinates associated with three-dimensional voxelated data values.

[0138] Returning to element 1804, in one embodiment of the present teachings, the second sensor is a non-imaging sensor that measures one or more intangible attributes in the real space. Representative intangible attributes that may exist in the real space include, but are not limited to radio signal, throughput of a connectivity signal, latency of the connectivity signal, interference of the connectivity signal, volatility of the connectivity signal, stability of the connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, temperature, energy, force field, dielectric constant, gases, and magnetic permeability. The present teachings recognize that when the intangible attribute is, for example, energy present in real space, one or more properties of energy are capable of being measured using non-imaging sensors. Examples of such properties may include, without limitation, frequency, wavelength, amplitude, signal strength (or signal power), phase angle, concentration, pressure, and phase difference. Intangible attributes may possess predefined properties depending on the intangible attribute sources generating them. For example, a radio signal may include a sinusoidal wave (or a cosinusoidal wave) having a predefined amplitude and phase at different instances in time.

[0139] In those instances where the second sensor measures gas concentrations, the second sensor is at least one sensor selected from a group including electrochemical sensor, infrared sensor, catalytic sensor, photoionization detection sensor, metal oxide semiconductor sensor, photoacoustic gas sensor.

[0140] The intangible attribute, in on embodiment of the present teachings, is a signal propagating in the real space. The sensor may measure the intensity of the signal. However, the present teachings are not so limited and may include other signal properties. The property of the signal propagating in the real space includes at least one property chosen from a group including frequency regime of the signal propagating in the real space, strength of the signal propagating in the real space, wavelength of the signal propagating in the real space, phase angle of the signal propagating in the space, phase difference of the signal propagating in the real space, angle of arrival of the signal propagating in the real space, peak frequency of the signal propagating in the real space, and direction of the signal propagating in the real space, and name of the source of the signal propagating in the real space.

[0141] The second sensor, in one implementation of the present teachings, is a software defined radio (“SDR”) that is physically attached to the eyewear (e.g., AR / VR headset) and is located at an offset distance from the fist sensor. As discussed above, the first sensor (e.g., an imaging sensor) is mounted on or integrated into the eyewear. In this exemplar implementation, the offset is static, in that the offset does not change during deployment of the eyewear when the first sensor and second sensor measure the first attribute and intensity values of one or more different intangible attributes, respectively. The offset distance may be determined as the distance, in three-dimensional space (i.e., X-coordinate, Y-coordinate, Z-coordinate) between first sensor and the second sensor. Using the three-dimensional coordinate where the first attribute was obtained and the static offset distance, the three-dimensional coordinates of intensity measurements may be obtained.

[0142] However, as discussed above, the present teachings are not so limited. In another implementation the offset is dynamic, and not static. In other words, offset distance between the first sensor and the second sensor changes at one or more three-dimensional coordinates in the real space. By way of example, the first sensor is be located on the eyewear that is positioned on a user’s head and the second sensor is located on a user’s hand. As the user moves with the real space, the offset distance between the first sensor and the second sensor in the user’s hand may change at one or more three-dimensional coordinates in the real space.

[0143] In one aspect of the present teachings, the second sensor includes or is coupled to components and / or modules for determine the three-dimensional coordinate of each measurement such as an IMU (e.g., IMU 442 of Figure 4) and one or more position sensors (e.g., one or more position sensors 435 of Figure 4), and a tracking module (e.g., tracking module 423 of Figure 4). A global positioning system (“GPS”) receiver and / or a Global Navigation Satellite System (“GNSS”) receiver may also be mounted onto of communicatively coupled to each eyewear to provide position, velocity, and timing information. Moreover, one or more position sensors and / or modules, described above, may be coupled to the second sensor to provide three- dimensional coordinates of the second sensor in real space.

[0144] In another aspect of the present teachings, obtaining an offset distance includes identifying, using one or more position sensors, one of more of three-dimensional coordinates of the second sensor in real space. Another element includes determining, using a tracking module, the spatial offset distance between one or more of the three-dimensional coordinates of thesecond sensor and one or more three-dimensional coordinates of the first sensor. The spatial offset distance may be, for example, a distance between the three-dimensional coordinates where the first attribute value is obtained and the corresponding three-dimensional coordinates of the second sensor.

[0145] In one embodiment of the present teachings, described in greater detail below, the second sensor, is an SDR that includes an emitter and receiver. The emitter emits an incident frequency at a location of examination or investigation and the receiver receives a reflected frequency. The reflected frequency and the reflected frequency intensity informs on one or more gases and gas concentrations present in the real space.

[0146] In one aspect of the present teachings, multiple second sensors are displaced in the real space of interest to obtain the second type of dataset. By way of example, to expedite collection of the second type of dataset, the first sensor and the second sensor are disposed on multiple eyewear. Each of the eyewear are displaced within the real space to collect intensity values of one or more different intangible attributes present in the real space. Preferably, the multiple second sensors, whether physically attached to the eyewear or separate from the eyecare are displaced within the real space simultaneously. In another embodiment of the present teachings, multiple second sensors are displaced within the real space at different periods of time.

[0147] In one embodiment of the present teachings, the second sensor obtains intangible attribute measurements (e.g., radio signal) that ranges from between about 20 kilohertz and about 75 gigahertz. In another embodiment of the present teachings, the second sensor obtain intangible attribute measurements that ranges from between about 30 megahertz and about 6 gigahertz.

[0148] In a preferred embodiment of the present teachings, the second sensor is a software defined radio sensor. The software defined radio sensor operates within a range of frequency that includes one or more frequency blocks (e.g., FB I and FB2 of Figure 12) and obtains measurements (e.g., 1202-1, 1202-2, 1202-3 of Figure 12) of an intangible attribute at different locations in real space, i.e., different values of X-, Y-, and Z-coordinates that define real space. The software defined radio sensor measures, for at least one of the frequency blocks, property values (e.g., Il, 12, 13, 14, and 15) at one or more of the three-dimensional coordinates in the real space.

[0149] The present teachings recognize that an intangible attribute (e.g., a radio signal) may be received or obtained from one or more intangible attribute signals at the same or different instants in time. Moreover, the present teachings recognize that for the intangible attribute received or obtained in a particular frequency block (e.g., FBI), the frequency measurements at the different X, Y, and Z-coordinates may be generated from two or more intangible attribute signals generating frequencies in this frequency regime, such that they generate: (1) overlapping frequency ranges and / or (2) two discrete, non-overlapping, frequency ranges.

[0150] Moreover, different types of intangible attribute signals, e.g., Wi-Fi™ source, Bluetooth™ source, and / or electric power generator, may produce an intangible attribute corresponding to different frequencies and / or signal strengths. Other examples of energy sources known in the art may also be contemplated for the intangible attribute signals. Stated another way, each of the different energy sources may be typically known to produce, or operate at, their own characteristic “frequency, range of frequency, or ranges of frequencies,” which are hereinafter referred to as “frequency regimes.”

[0151] Each non-imaging sensor may be configured to measure the signal intensity (or amplitude) of the received radio signal, within a frequency block (e.g., measures signal amplitude at a frequency that is between about 2401 megahertz and about 2,423 megahertz). In yet another embodiment of the present teachings, a non-imaging sensor may measure one or more properties in multiple frequency blocks. By way of example, a non-imaging sensor may measure one or more properties of an intangible attribute within a particular frequency block and then measure, at a different period of time, one or more properties of another or the same intangible attribute within another frequency block.

[0152] The type of frequency regimes produced from or by such energy sources may provide insight into whether an energy source is operating properly or malfunctioning. For example, malfunctioning power transformers may emit characteristic radio signals that differ from those emitted when operating as intended. As a result, the detection of such characteristic radio signals is insightful in the identification and / or a location of a malfunctioning power transformer present in a real space. Other examples may include identifying an energy source (or the intangible attribute signal) in the real space based on emission of radio signals therefrom at frequency regimes (or signal strengths) different from those known or expected within a preset proximity of the eyewear.

[0153] The intangible attribute signal, in one embodiment of the present teachings, acts as an access point, in that the intangible attribute signal may facilitate creating a wireless local area network (WLAN) and allows for wireless-capable devices (z.e., wireless clients) to connect to the WLAN. Obtaining a data packet that provides identifying information about the intangible attribute signal acting as an access point, in one embodiment of the present teachings, includes an element of transmitting, from the eyewear or a device coupled to the eyewear, a probe request management frame (hereinafter “probe request”) data packet to discover wireless networks within proximity to the eyewear or a device coupled to the eyewear. The probe request data packet may be sent passively, within one frequency range or band, or multiple frequency blocks, to identify any wireless network. The probe request data packet may also be sent to identify a particular wireless network having a unique service set identifier (“SSID”) The probe request data packet may include information related to supported data rates, MAC address, SSID, and wireless networking standards (e.g., an IEEE 802.11 wireless networking standard.).

[0154] Element 1804, in another embodiment of the present teachings, includes obtaining, from the intangible attribute signal acting as an access point, a probe response frame (hereinafter “probe response”) data packet. The probe response data packet provides identifying information such as the MAC address, SSID of the intangible attribute signal acting as an access point, supported data rates, encryption types, if required, and wireless networking standards.

[0155] The present teachings recognize that, at regular intervals (e.g., once every 0.1 seconds), access points may transmit a beacon frame data packet that may include identifying information, for example, MAC address, SSID, and / or supported data transfer rate. In another embodiment of the present teachings, obtaining a data packet that provides identifying information about the intangible attribute signal acting as an access point includes an element of obtaining a beacon frame data packet from the intangible attribute signal acting as an access point. As discussed above, identifying information from the beacon frame data packet may be used to generate a second type of dataset.

[0156] In one embodiment of the present teachings, the intangible attribute data set is generated using the identifying information. By way of example, in Figure 12 the column header named “Measured Property Value” may include identifying information, obtained from the data packet, such as a MAC address or a SSID instead of a frequency regime. In the present application, “attribute value” and “property value” are used interchangeably in the context of their broadestmeanings. The “attribute value” may refer to a value of a property (e.g., frequency regime, signal strength, phase angle, wavelength, amplitude, power, data packet identifier, data source identifier, etc.) of an intangible attribute, such as radio signals.

[0157] In some real -world scenarios, the intangible attribute signal may not be an access point but a device that is capable of connecting to or is connected to a WLAN (z.e., a wireless client). In other words, the intangible attribute signal is a client (e.g., a phone, camera, or audio recording device) that is connected to or is capable of connecting to a WLAN created by an access point. In one embodiment of the present teachings, the element of obtaining, from an intangible attribute signal, a data packet that provides identifying information about the intangible attribute signal includes obtaining the probe request data packet that is transmitted from an intangible attribute signal acting as a client. As discussed above, the probe request data packet includes identifying information such as a MAC address or SSID.

[0158] The non-imaging sensor may be additionally configured to measure a property of one or more different types of intangible attributes at a predetermined measuring frequency (e.g., a measuring frequency of 100 measurements per second). Accordingly, the processor may store various intangible attribute properties (e.g., phase difference, wavelength, peak frequency, signal strength, etc.) and their respective values of one or more of the intangible attributes (e.g., radio signal) in the computer memory and create the second type of dataset. The processor may determine the intangible attribute measurement at various locations (or 3D coordinates) in the real space based on a spatial movement of the eyewear therein. To minimize memory usage, processing load, and / or energy usage, however, not all of the sampled or obtained property measurements may be used in element 1804. Rather, in one embodiment of the present teachings, an optional element includes selecting one or more property measurements of one or more different types of intangible attributes at a frequency (e.g., a selecting rate of 10 measurements per second) that is less than the measuring frequency at which the sensor measures one or more of the properties.

[0159] In yet another embodiment of the present teachings, element 1804 includes performing interpolation to predict a location and / or direction of the location of intangible attribute signal. Interpolating may be implemented, for example, when an intangible attribute signal is disposed within a confined region. One or more of the second sensors, mounted on or communicatively coupled to an eyewear, receive intangible attribute measurements outside of the confined regionsbut is not able to obtain intangible attribute measurements from within the confined region. Using interpolation, the location and / or direction of the location of intangible attribute signal, within the confined region, may be predicted.

[0160] In this embodiment, two or more portions of the second type of dataset and / or the frequency domain dataset associated with a particular type of intangible attribute signal are used to compute a predicted intangible attribute measurement associated with the particular type of intangible attribute signal. As discussed above, each portion of the second type of dataset and / or the frequency domain dataset is associated with one or more of three-dimensional location coordinates. A predicted intangible attribute measurement associated with the particular type of intangible attribute signal, in one embodiment of the present teachings, is determined by calculating an intermediate positioned on a linear trajectory that intersects two or more of the corresponding three-dimensional location coordinates.

[0161] In certain aspects of the present teachings, the three-dimensional location coordinate associated with the intangible attribute signal located within the confined region is relative to one or more three-dimensional coordinates that define a perimeter or boundary of the confined region.

[0162] In yet another embodiment of the present teachings, element 1804 includes performing extrapolation to predict a location and / or direction of the location of intangible attribute signal. Interpolation may be implemented, for example, when an intangible attribute signal is disposed outside of the real space (e.g., outside the line of sight of one or more non-imaging sensor and / or imaging sensors). Using extrapolation, the location and / or direction of the exterior location of intangible attribute signal, beyond the real space, may be predicted. A predicted intangible attribute measurement associated with the particular type of intangible attribute signal, in one embodiment of the present teachings, is determined by calculating an exterior positioned on a linear trajectory that intersects two or more of corresponding three-dimensional location coordinates.

[0163] The present teachings recognize that storing, in the computer memory, and processing, using one or more processors on the eyewear or off the eyewear, one or more second type of datasets may be undesirable due to the quantity of data in each second type of dataset. At different locations in real space, i.e., different values of X-, Y- and Z-coordinates that define real space, the plurality of sensors may receive duplicative measured values. Therefore, the presentteachings contemplate methods for filtering the second type of dataset to produce a filtered second type of dataset. Filtering, in one embodiment of the present teachings, includes removing, within the second type of dataset, duplicative data that is spatially and / or temporally substantially similar.

[0164] By way of example, one or more duplicative data, within a second type of dataset, that does not vary (within a tolerable range) spatially may be removed to reduce the size of the second type of dataset. In other words, for a particular instance in time, if measurements for different locations in real space, identify the same intangible attribute properties (e.g., properties such as frequency regime or MAC address and frequency amplitude), and measure the same intangible attribute amplitude, then duplicative intangible attribute measurements may be deleted, or at least discarded from a task / data processing queue.

[0165] By way of another example, one or more duplicative data, within a second type of dataset, that does not vary temporally (within a tolerable range) may be removed. For example, if the measurements for the intangible attribute property (e.g., frequency regime, MAC address, or frequency amplitude) at a particular location in real space (e.g., XI, Yl, Zl) remains the same at different instances in time (e.g., at t=Tl, T2 and T3)), then duplicative data may be removed.

[0166] By way of yet another example, for a particular location of the eyewear (or the nonimaging sensors) in real space (e.g., XI, Yl, Zl), the property values of an intangible attribute may vary within the same time period or multiple time periods. Slight varying of frequency values may result due to interference from, for example, radio signals from external devices such as other electronics such as microwaves ovens, tablets, computers, computer peripherals, phones, distant routers, MRI machines, power lines, cordless telephones, smart meters. The present teachings allow for these variable measurements to be identified as a single intangible attribute, for example, a particular frequency value. In one embodiment of the present teachings, a mean is determined for all measurements of frequency block or range. The frequency regime (e.g., the signal frequency emitted by intangible attribute signal) is the mean frequency value. In one embodiment of the present teachings, each measured frequency data that is within three standard deviations of the mean is replaced with the mean frequency value. In a preferred embodiment of the present teachings, each measured frequency data that is within two standard deviations of the mean is replaced with the mean frequency value. In a more preferred embodiment of the presentteachings, each measured frequency data that is within one standard deviation of the mean is replaced with the mean frequency value.

[0167] By way of yet another example, one or more of the frequency blocks that do not have one or more of the measured intensity values within the frequency block are removed from the second type of dataset.

[0168] The present teachings also recognize that one or more second type of datasets for one or more real spaces may be stored in memory for use at a later time. In one embodiment of the present teachings, obtaining including obtaining or accessing a pre-stored second type of dataset including one or more intangible attributes, such as radio signals, and / or various measurements related thereto.

[0169] As discussed above, element 1686 includes performing, for the frequency block, a fast Fourier transformation on the intensity values of one or more different intangible attributes to produce a frequency domain dataset that includes one or more frequency peak values and the three-dimensional coordinates of the frequency peak values.

[0170] The present teachings recognize that a fast Fourier transformation is but one exemplar mathematical transformation that transforms a spatial and / or temporal dataset to frequency domain dataset. The present teachings contemplate use of at least one mathematical transformation chosen from a group including fast Fourier transform, discrete Fourier transform, and discrete cosine transform.

[0171] In a preferred embodiment of the present teaching, a fast Fourier’s transform is used to mathematically transform the second type of dataset set to produce the frequency domain dataset. By way of example, a fast Fourier’s transform may be performed on one or more of the properties (e.g, intensity) measured and the corresponding different instances in time of measurement to produce the frequency domain dataset. The measured properties, in this implementation, is the measured property represented in a temporal frequency domain. In a preferred embodiment of the present teachings, each of the measured properties is a temporal frequency pattern, which is discussed in greater detail below.

[0172] By way of another example, a fast Fourier’s transform may also be performed on one or more of the properties measured and the corresponding one or more three-dimensional coordinates of measurement to produce a frequency domain dataset in a spatial frequency domain. As a result, the frequency domain dataset may provide information regarding one ormore of the measured properties in the temporal frequency domain and / or the spatial frequency domain.

[0173] In one embodiment of the present teachings, data in the frequency domain dataset is substantially similar to data residing in the second type of dataset. For example, the different instances in time of measurement in the second type of dataset are the different instances in time the measurements were made by one or more non-imaging sensors, one or more of the three- dimensional coordinates of one or more frequency peak is the three-dimensional locations where the measurements were made by one or more of the non-imaging sensors

[0174] In one embodiment of the present teachings, the second sensor measures a radio signal and the processor converts each obtained radio signal from the second type of dataset into a temporal frequency domain and / or a spatial frequency domain. By way of example, one or more second sensors may receive a measurement of signal intensity (e.g., strength) for a first radio signal XI and a second radio signal X2 (collectively referred to as radio signals XI, X2) in time domain at a given time instant, t=Tl for a particular frequency block, which is included in second type of dataset. A processor, to produce the frequency domain dataset, may mathematically transform each of the received radio signals, such as radio signals XI and X2, into a corresponding spatial frequency pattern and / or a temporal frequency pattern, TF1 and TF2, respectively.

[0175] A temporal frequency pattern may include multiple temporal frequency peaks. Each of these peaks may correspond to a temporal frequency at which the radio signal may have a signal power (or signal strength) greater than a predefined temporal frequency strength threshold value. The processor may be configured to ignore and / or set as noise those temporal frequencies at which the radio signal may have a signal power (or signal strength) below or less than such predefined threshold value. In some examples, the transformed signal (in frequency domain) may include one or more frequency blocks excluding a peak above the predefined threshold value.

[0176] The present teachings recognize that, at different instances in time (e.g., at t=Tl, T2 and T3), the temporal frequency pattern for an intangible attribute may vary. By way of example, the temporal frequency value and / or temporal strength value for each of the one of more temporal frequency peaks, included in the temporal frequency pattern, may vary at different instances in time. Similarly, at different locations in real space, the spatial frequency pattern for an intangible attribute may vary. The spatial frequency value and / or spatial strength value for each of the oneor more spatial frequency peaks included in the spatial frequency pattern, for example, may vary from one location of measurement in real space to another location.

[0177] To reduce computational complexity, memory usage and / or the amount of data in the frequency domain dataset, it may be desirable to identify variable temporal frequency patterns and / or variable spatial frequency patterns as the temporal frequency pattern and / or spatial frequency of a particular intangible attribute or a particular type of intangible attribute signal. The present teachings, therefore, provide an optional identifying element for identifying variable temporal frequency patterns and / or variable spatial frequency patterns as the temporal frequency pattern and / or spatial frequency of a particular intangible attribute or a particular type of intangible attribute signal.

[0178] In one embodiment of the present teachings, the optional identifying element includes calculating, for an intangible attribute, a mean temporal frequency pattern from variable temporal frequency patterns. The mean temporal frequency pattern may include one or more mean temporal frequency peaks, each having a mean temporal frequency value and / or a mean temporal strength value. In one embodiment of the present teachings, each temporal frequency pattern that is within three standard deviations of the mean temporal frequency pattern (e.g., the temporal frequency value and / or temporal strength value, for one or more temporal frequency peaks of the temporal frequency pattern, is within three standard deviation of the mean temporal frequency value and / or a mean temporal strength value of the mean temporal frequency peaks of the mean temporal frequency pattern) is replaced with the mean temporal frequency pattern. In a preferred embodiment of the present teachings, each temporal frequency pattern that is within two standard deviations of the mean temporal frequency pattern is replaced with the mean temporal frequency pattern. In a more preferred embodiment of the present teachings, each temporal frequency pattern that is within one standard deviation of the mean each temporal frequency pattern is replaced with the mean temporal frequency pattern.

[0179] In another embodiment of the present teachings, for an intangible attribute having variable spatial frequency patterns, a mean spatial frequency pattern is determined. The mean spatial frequency pattern may include one or more mean spatial frequency peaks. Each mean spatial frequency peak may include a mean spatial frequency value and / or a mean spatial strength value. In one embodiment of the present teachings, each spatial frequency pattern that is within three standard deviations of the mean spatial frequency pattern (e.g., the spatial frequencyvalue and / or spatial strength value, for one or more spatial frequency peaks of the spatial frequency pattern, is within three standard deviation of the mean spatial frequency value and / or a mean spatial strength value of the mean spatial peaks of the mean spatial frequency pattern) is replaced with the mean spatial frequency pattern. In a preferred embodiment of the present teachings, each spatial frequency pattern that is within two standard deviations of the mean spatial frequency pattern is replaced with the mean spatial frequency pattern. In a more preferred embodiment of the present teachings, each spatial frequency pattern that is within one standard deviation of the mean each spatial frequency pattern is replaced with the mean spatial frequency pattern.

[0180] Another optional element includes removing or filtering duplicative data after performing the mathematical transformation. As a result, the size of the source identifying data set may be reduced. In one embodiment of the present teachings, duplicative information (e.g, one or more spatial frequency values, one or more spatial strength values, one or more temporal frequency values, and / or temporal strength values) derived from spatial frequency patterns and / or temporal frequency patterns that are associated a particular type of intangible attribute is deleted to create the frequency domain dataset. The spatial frequency patterns and / or temporal frequency patterns may be the mean spatial frequency and / or temporal frequency pattern for a particular type of intangible attribute or particular type of intangible attribute signal.

[0181] By way of example, for a particular location in real space (e.g., XI, Yl, Zl), if after performing the mathematical transformation, dataset includes multiple data entries for a particular type of intangible attribute, and if each of these includes a temporal frequency value of 1,300 Megahertz and a temporal signal strength of -70 decibel-milliwatts, then one data entry remains in the frequency domain dataset and the remaining duplicative data is fdtered (e.g., deleted or discarded) from a task / data processing queue. This process is commonly known as deduplication in the art.

[0182] As discussed above, element 1808 includes identifying, based on one or more of the frequency peak values and without using the first type of dataset, one or more of the intangible attribute signals.

[0183] In one embodiment, a single frequency peak value may be used to identify or determine that an intangible attribute signal is present in the real space. By way of example, an intangible attribute signal generated by engineered source e.g., WIFI, mobile phone, radio station signal,emergency beacon, or GPS satellite) transmits at a specific, known frequency or frequency block. In one embodiment of the present teachings, performing element 1806 includes producing a frequency domain dataset using a single frequency peak having a single frequency peak value. Where there is a single frequency peak, identifying includes ascertaining whether a frequency peak intensity value associated with the single frequency peak value equals or exceeds a predetermined intensity threshold value assigned to the frequency block in which the single frequency peak is obtained. If the frequency peak intensity value equals or exceeds a predetermined intensity threshold value, an intangible attribute signal is identified as present in the real space. If the frequency peak intensity value is below the predetermined intensity threshold value, the single frequency peak is regarded as noise and is ignored.

[0184] Another element may include identifying the intangible attribute signal by matching single frequency peak value with a frequency peak value of a known intangible attribute signal and / or matching frequency block in which the single frequency peak value resides with frequency block of a known intangible attribute signal.

[0185] The present teachings also recognize that the single frequency peak may be one frequency peak of multiple frequency peaks that represent a spatial and / or temporal frequency peak pattern of a known intangible attribute signal. Thus, the present teaching enables identification of an intangible attribute present in real space using a single frequency peak. For example, if the single frequency peak, among multiple frequency peaks, is in a frequency block that is associated with a known engineered source (z.e., a WIFI signal) and the frequency peak value equals or exceeds a predetermined intensity threshold value, the intangible attribute signal is identified as a WIFI signal.

[0186] In another embodiment, multiple frequency peaks may be used to identify one or more intangible attribute signal is present in the real space. The present teachings recognized that some intangible attribute signals (e.g, a gas signal associated with a gas), which may not be signals generated from engineered sources, cannot be identified by a single frequency peak. Multiple different intangible attribute signals, for example, may exist within a frequency block or have the same or similar frequency peak value. Therefore, multiple frequency peaks may be used to identify one or more different intangible attribute signals inside the real space.

[0187] Identifying the presence of one or more of the intangible attribute signals, in one embodiment of the present teachings, includes matching, within a predefined tolerance, two ormore of the multiple frequency peaks with a spatial reference pattern generated by an intangible attribute signal and / or a temporal reference pattern generated by the intangible attribute signal to identify the intangible attribute signal.

[0188] The present teachings recognize that, in certain embodiments, a particular type of intangible attribute generates and / or is associated with a spatial frequency reference pattern and / or a temporal frequency reference pattern. In other words, a particular type of intangible attribute generates and is associated with one or more spatial frequencies distributed in a known frequency pattern and / or one or more temporal frequencies distributed in a known frequency pattern. The spatial reference pattern and / or temporal reference pattern may be obtained, for example, by applying a mathematical transformation, e.g., a fast Fourier’s transform, to a signal generated by a particular type of known intangible attribute signal.

[0189] The temporal reference pattern may include one or more reference temporal frequency peaks and each of the reference temporal frequency peaks has a reference temporal frequency value. Matching at least the part of the frequency domain dataset with a temporal reference pattern, in one embodiment of the present teachings, includes matching, within the predefined tolerance, multiple temporal frequency peaks values with one or more of the reference temporal frequency values associated with the temporal reference pattern.

[0190] The spatial reference pattern includes one or more reference spatial peaks and each of the reference spatial frequency peaks has a reference spatial frequency value. Matching at least the part of the frequency domain dataset with a spatial reference pattern, in another embodiment of the present teachings, includes matching, within the predefined tolerance, multiple spatial frequency peak values with one or more of the reference spatial values associated with the spatial reference pattern.

[0191] The spatial reference pattern and / or the temporal pattern may be acquired by the processor for each of the particular types of intangible attribute signals sought to be identified. In other words, each type of, intangible attribute signal, e.g., energy source “associated with” a spatial reference pattern and / or the temporal reference pattern that may be stored in the computer memory. When at least part of the frequency domain dataset matches or substantially matches a known spatial reference pattern and / or the temporal reference pattern, then the intangible attribute signal associated with the known spatially distributed reference pattern and / or theknown temporally distributed reference pattern may be identified to be present in the real space by the processor.

[0192] In yet another embodiment of the present teachings, the frequency domain dataset includes one or more spatial patterns and / or one or more temporal patterns. Each of the temporal patterns includes one or more temporal peaks, each having a temporal value e.g., 1 second) and a temporal strength value (e.g., -70 decibel-milliwatts). The temporal strength value is identified as being a strength value that is greater than a preceding temporal strength value and subsequent temporal strength value. Each of the spatial patterns includes one or more spatial peaks, each having a spatial value e.g., 1 meter) and a spatial strength value (e.g., -70 decibel-milliwatts). The spatial strength value is identified as being a strength value that is greater than a preceding spatial strength value and subsequent spatial strength value.

[0193] Element 1808, in one embodiment of the present teachings, includes comparing, within the frequency domain dataset, one or more spatial strength values of different spatial patterns and / or one more temporal strength values of different temporal patterns. Each of the spatial strength values and / or temporal strength values that are different than others of one or more of the intangible attribute strength values corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0194] In another embodiment of the present teachings, element 1808 includes determining whether one or more of the temporal values, in each of the one or more temporal patterns in the frequency domain dataset, is within a predetermined target temporal band and / or one or more spatial values of one more of the spatial patterns, is within a predetermined target spatial band. Each of the temporal patterns having one more of the temporal values that are within the predetermined target temporal band and / or each of the spatial patterns having one or more of the temporal values that are within the target spatial band corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space,

[0195] In yet another embodiment of the present teachings, element 1808 includes determining whether one or more of the temporal values, of one or more of the temporal patterns match a target temporal value and / or one or more of the temporal values, of one or more of the spatial patterns, matches a target spatial value. Each of the temporal patterns having one more of the temporal values that match the target temporal value and / or each of the spatial patterns havingone or more of the temporal values that match the target spatial value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0196] In yet another embodiment of the present teachings, element 1808 includes comparing, one or more temporal strength values of one or more of the temporal patterns, to a predetermined temporal strength threshold value, and / or one or more of the spatial strength values, of one or more spatial patterns, to a predetermined spatial strength threshold value. Each of the one or more temporal patterns having one or more temporal strength values that is greater than or equal to the predetermined temporal strength threshold value and / or each of one more of spatial patterns having one or more spatial strength values that is greater than or equal to the predetermined spatial strength threshold value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0197] When two or more intangible attribute signals, present in real space, operate in frequency regimes that include overlapping frequency ranges, the frequency domain dataset may be analyzed to identify the resulting two or more intangible attribute signals that may be present in an overlapping range. In preferred embodiments, the present teachings offer a differentiating technique to identify one or more frequencies in the overlapping range to determine which intangible attribute signals are generating a signal in the frequency regime.

[0198] As discussed above, in one embodiment of the present teachings, a mathematical transformation (e.g., a fast Fourier’s transform) is used to transform the filtered second type of dataset set to produce a frequency domain dataset in the spatial frequency domain and / or the temporal frequency domain. The frequency domain dataset may include one or more spatial frequency patterns and / or one or more temporal frequency patterns. Each of the temporal frequency peaks includes a temporal frequency value (e.g., 1,300 megahertz) and a temporal frequency strength value (e.g., -70 decibel-milliwatts) that is greater than a preceding and subsequent temporal frequency strength value. Each of the spatial frequency patterns includes one or more spatial frequency peaks, each having a spatial frequency value (e.g., 1 meter-1) and a spatial frequency strength value (e.g., -70 decibel-milliwatts). The spatial frequency strength value is identified as being a frequency strength value that is greater than a preceding spatial frequency strength value and subsequent spatial frequency strength value.

[0199] Element 1808, in one embodiment of the present teachings, includes comparing, within the frequency domain dataset, one or more spatial frequency strength values of different spatial frequency patterns and / or one more temporal frequency strength values of different temporal frequency patterns. Each of the spatial frequency strength values and / or temporal frequency strength values that is different than other spatial frequency strength values and / or temporal frequency strength values corresponds to and are identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0200] By way of example, the frequency domain dataset may include temporal frequency patterns, TF1 and TF2, that correspond to radio signals XI and X2, respectively. The processor may identify, in the temporal frequency pattern, TF1, a temporal frequency peak with the highest temporal frequency strength value, SI, and a corresponding temporal frequency value, Fl. Similarly, the processor may identify, in the temporal frequency pattern, TF2, a temporal frequency peak with the highest temporal frequency strength value, S2, and a corresponding temporal frequency value, F2.

[0201] The processor may compare the temporal frequency strength value SI and S2, of the received radio signals XI and X2, to differentiate between them. For example, if temporal frequency strength value SI is different than temporal frequency strength value S2, then temporal frequency strength value SI corresponds to and is identified as an intangible attribute signal that produces radio signal XI inside the real space. Temporal frequency strength value S2 corresponds to and is identified as another or different intangible attribute signal that produces radio signal X2 inside the real space.

[0202] In another embodiment of the present teachings, element 1808 includes determining whether one or more of the temporal frequency values, in each of the one or more temporal frequency patterns in the frequency domain dataset, is within a predetermined target temporal frequency block and / or one or more spatial frequency values of one more of the spatial frequency patterns, is within a predetermined target spatial frequency block. Each of the temporal frequency patterns that includes one more of temporal frequency values that is within the predetermined target temporal frequency block and / or each of the spatial frequency patterns that includes one or more spatial frequency values that is within the target spatial frequency block corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0203] By way of example, the processor may be configured to differentiate between the radio signals XI and X2 received at a given time instant based on a comparison between their respective temporal frequency values within a target temporal frequency block. For example, the processor may be preconfigured or dynamically configured with a target temporal frequency block ranging from 2000 megahertz to 2500 megahertz. As discussed above, the processor may identify, in the temporal frequency pattern, TF1, a temporal frequency peak with the highest temporal frequency strength value, SI, and a corresponding temporal frequency value, Fl. Similarly, the processor may identify, in the temporal frequency pattern, TF2, a temporal frequency peak with the highest temporal frequency strength value, S2, and a corresponding temporal frequency value, F2, as discussed above.

[0204] In a first example, temporal frequency value Fl may be located within the target frequency block, and the temporal frequency value F2 may be located outside the target frequency block. In this first case, the processor may be configured to select temporal frequency Fl, being located in (or belonging to) the target frequency block, as corresponding to and being identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0205] In a second example, if both the temporal frequency values Fl and F2 may be located within the target temporal frequency block, as radio signals XI and X2 are identified as intangible attribute signals that produce the intangible attributes inside the real space.

[0206] In yet another embodiment of the present teachings, element 1808 includes determining whether one or more of the temporal frequency values, of one or more of the temporal frequency patterns matches a target temporal frequency value and / or one or more of the spatial frequency values, of one or more of the spatial frequency patterns, matches a target spatial frequency value. Each of the temporal frequency patterns that includes a temporal frequency value that matches the target temporal frequency value and / or each of the spatial frequency patterns that includes a spatial frequency value that matches the target spatial frequency value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0207] By way of example, the processor may be preconfigured or dynamically configured with a target temporal frequency value of 900 megahertz. The processor compares each of the temporal frequency values Fl and F2 with the target temporal frequency value. If the temporalfrequency value Fl and / or F2 match the target temporal frequency value, then the radio signal XI and / or X2, respectively, corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0208] In yet another embodiment of the present teachings, element 1808 includes comparing, one or more temporal frequency strength values of one or more of the temporal frequency patterns, to a predetermined temporal frequency strength threshold value, and / or one or more spatial frequency strength values, of one or more spatial frequency patterns, to a predetermined spatial frequency strength threshold value. Each of one or more temporal frequency patterns that includes one or more temporal frequency strength values that is greater than or equal to the predetermined temporal frequency strength threshold value and / or each of one more of spatial frequency patterns that includes one or more spatial frequency strength values that is greater than or equal to the predetermined spatial frequency strength threshold value corresponds to and is identified as one of one or more of the intangible attributes sources that produce the intangible attributes inside the real space.

[0209] In yet another embodiment of the present teachings, the processor is configured to distinguish between the radio signals received in the same or differing frequency regimes based on additional properties associated with the radio signals. Examples of the additional properties may include, but are not limited to, modulation schemes, signal-to-noise ratio (SNR), bit error rate (BER), and collision detection protocols. Examples of the collision detection protocols may include, but are not limited to, carrier sense multiple access with collision avoidance (CSMA / CA) protocol and adaptive frequency hopping (AFH) protocol.

[0210] By way of example, the processor may distinguish between and / or identify the received radio signals based on an underlying modulation scheme to select only a subset of one or more radio signals, e.g., for determining the corresponding intangible attribute signal. By way of another example, the processor may be configured to select only those received radio signals whose modulation parameters may pertain to a predetermined modulation scheme such as Orthogonal Frequency Division Multiplexing (OFDM) scheme. For example, the processor may determine that the received radio signals may be generated by a Wi-Fi™ source (or Wi-Fi™- enabled source) based on the received radio signals being modulated (or implementing) DSSS or OFDM scheme. In another example, the processor may determine that the received radio signals may be generated by a Bluetooth™ source (or Bluetooth™-enabled source) based on thereceived radio signals being modulated (or implementing) GFSK. By way of yet another example, the processor may implement any suitable time division multiplexing (TDM) algorithm known in the art to produce a multiplexed radio signal (in time domain) from the received radio signals (in time domain) having similar modulation parameters. The multiplexed radio signal may be converted into frequency domain to process the same type of radio signals as a batch for determining the highest frequency peak (and related aspects such as frequency value and frequency strength value) therein, as discussed above.

[0211] In another embodiment of the present teachings, information from a received data packet may assist the processor in identifying the presence of intangible attribute signals that produce the intangible attributes inside the real space. By way of example, a MAC address provided as part of the data packet identifies an intangible attribute signal associated therewith.

[0212] Returning now to element 1810, which includes spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a fist type of spatialized dataset and a spatialized offset distance. Spatializing is preferably implemented using a three-dimensional spatializing module. In preferred embodiments of the present teachings, an image spatializing module (e.g., image spatializing module 576 of Figure 5A or image spatializing module 576' of Figure 5B) is used to create the first type of spatialized dataset. The image spatializing module may reside on the eyewear or on an external processor. If the image spatializing module resides on the external processor, then the external processor, which is external to the eyewear, is communicatively coupled to the eyewear to convey to it the “processed information,” resulting from processing of the spatialized image value dataset.

[0213] In those embodiments where an image spatializing module is used in element 1810, this module begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the image spatializing module integrates the subdivisions to create a spatialized model of the real space. In the spatializing element 1810, the image values may be distributed, based upon the spatialized model, to create the first type of spatialized dataset. Further, “real space,” in the present methods, is electronically represented using a plurality of the three-dimensional location coordinates, which may have been retrieved from obtaining elements 1804.

[0214] The offset distance is similarly spatialized by a spatializing module. A three-dimensional offset distance, in real space, between the first sensor and the second senor is spatialized to createthe spatialized offset distance. In one implementation of the present teachings, where the offset distance is static and does not change, the spatial offset distance is also static. However, if the offset distance, in real space, is dynamic and not static, the spatialized offset distance is different between the first sensor and the second sensor.

[0215] Method 1800 further includes element 1812, which includes spatializing, using the plurality of the three-dimensional coordinates, the second type of dataset to create a second type of spatialized dataset. In preferred embodiments of the present teachings, an attribute spatializing module (e.g., attribute spatializing module 578 of Figure 5 A or attribute spatializing module 578' of Figure 5B) is used to create the spatialized frequency peak values. The attribute spatializing module may reside on the eyewear or on an external processor. If the attribute spatializing module resides on the external processor, then the external processor, which is external to the eyewear, is communicatively coupled to the eyewear to convey to it the “processed information,” resulting from processing of one or more spatialized frequency peak values.

[0216] In those embodiments where an attribute spatializing module is used in element 1812, this module, like the image spatializing module of element 1810, begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the attribute spatializing module integrates the subdivisions to create a spatialized model of the real space. In spatializing element 1812, one or more of the frequency peak values may be distributed, based upon the spatialized model, to create one or more of the spatialized frequency peak values.

[0217] In one implementation of method 1800, if a spatialized model is created using an image spatializing module or an attribute spatializing module, then the other spatializing module is free to use the created spatializing model, without the need for creating another or recreating a spatialized model to effect distribution of the data within the image value dataset or frequency domain dataset, as the case may be.

[0218] After spatializing of elements 1810 and 1812, described in greater detail below, produce the first type of spatialized dataset and the second type of spatialized dataset and the spatialized imaging value dataset, then method 1800 proceeds to element 1814 which includes aligning, using the spatialized offset distance, the first type of spatialized dataset with the second type of spatialized dataset to create an enhanced three-dimensional spatialized environment.

[0219] In preferred embodiments of the present teachings, an aligning module (e.g., an aligning module 580 of Figure 5A or an aligning module 580' of Figure 5B) is used to carry out aligning element 1814. The aligning module may reside on an eyewear or on an external processor, which is disposed external to and communicatively coupled to the eyewear. In method 1800, the enhanced three-dimensional spatialized environment represents the “processed information” that is subsequently rendered on a display component of an eyewear, user device, or a smartphone.

[0220] Regardless of where it is located, the aligning module may align, using the spatialized offset distance, cartesian coordinate of the first type of spatialized dataset and the second type of spatialized dataset. Using the offset distance and one or more common vertices, found in both the first type of spatialized dataset and the second type of spatialized dataset, to effect alignment represents another embodiment of element 1814

[0221] After completing the alignment, the first type of spatialized dataset with one or more of the spatialized frequency peak values, element 1814 proceeds to element 1816, which includes rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment. One or more of the intangible attribute signals, associated with one or more of the frequency peak values, are visually represented in the enhanced three-dimensional spatialized environment.

[0222] A rendering element (e.g., rendering engine 582 of Figure 5A or rendering engine 582’ of Figure 5B) may render the enhanced three-dimensional spatialized environment on a display component (e.g., VO 417 of Figure 4). In certain exemplary embodiments, the display component is integrated into a client device, which is at least one device chosen from a group comprising wearable, smartphone, and computer. In preferred embodiments of the present teachings, the wearable is a laptop.

[0223] Regardless of where the display component is located, the rending element may render multiple intangible attribute signals present in the real space. In one embodiment, rendering the enhanced three-dimensional spatialized environment further includes an element of visually representing multiple intangible attribute signals in the enhanced three-dimensional spatialized environment. Each of the intangible attribute signals are presented to be selectable. By way of example, a list of the multiple intangible attribute signals present in the real space may be presented in a collapsable window within the virtual space. A user, for example, can virtually select an intangible attribute signal the user want to be rendered.

[0224] Another element includes receiving selection one of the intangible attribute signals by the user. The user, for example, may virtually touch the intangible attribute signal that it wants to render. Upon receiving selection, another element includes rendering, in the virtual space in the AR / VR headset using the rendering engine, and within the enhanced three-dimensional spatialized environment, the selected intangible attribute signal.

[0225] Preferably, the selected intangible attribute signal is rendered at spatialized locations that correspond to locations of the three-dimensional coordinates of the frequency peak values that are associated with the selected intangible attribute signal. By way of example, each signal strength indicators 1612 of Figure 16 is rendered at spatialized locations that correspond to locations of the three-dimensional coordinates of the frequency peak values for a selected intangible attribute signal.

[0226] In one aspect, the intensity of the selected intangible attribute signal at one or more of the spatialized locations is also rendered. Visual representation of the intensity of a rendered intangible attribute signal allows a user, using the eyewear, to quickly and easily identity the signal distribution through the real space that would otherwise be invisible to the user. In a preferred embodiment the intensity of an intangible attribute signal is virtually represented by a virtual object (e.g., a bar).

[0227] In another aspect of the present teachings, the frequency domain dataset includes, for each of one or more of the frequency peak values, an associated frequency peak intensity value. The signal intensity of the intangible attribute signal may be the frequency peak intensity value of one of the frequency peaks associated with the intangible attribute signal. In rendering the virtual object e.g., a virtual bar), a size of the virtual object corresponds to the frequency peak intensity value of one of the frequency peaks associated with the intangible attribute signal.

[0228] To this end, another element includes visually representing a signal intensity of the selected intangible attribute signal with a virtual object at one or more of the spatialized locations in the enhanced three-dimensional spatialized environment.

[0229] Another element includes choosing, by a user, the virtual object at one of the spatially distributed locations. Following the choosing element, a virtual representing element is carried out, which includes virtually representing, within the enhanced three-dimensional spatialized environment, an intensity value of the selected virtual object.

[0230] As discussed above, interpolating and / or extrapolating techniques may be used to generate intermediate frequency peak value between known frequency peak values and predicted frequency peak values, respectively. This enables virtual representation of an intangible attribute signal at locations where intensity values of one or more different intangible attributes were not collected by the second sensor. Thus, even without second sensor measurement, the enhanced three-dimensional spatialized environment provides a relatively accurate representation an intangible attribute signal propagating through a real space.

[0231] When extrapolation techniques are used in element 1804, as discussed above, element 1816 of displaying and / or the causing to display includes presenting an exterior virtual object at a corresponding interior three-dimensional location coordinate associated with the particular type of the intangible attribute signal. The exterior virtual objects, for example, includes arrows and / or an illustration of the particular type of the intangible attribute signal.

[0232] Rendering, in another aspect of the present teachings, includes rendering exterior virtual objects at exterior object locations, in the enhanced three-dimensional spatialized environment. The exterior object locations correspond to locations of the corresponding exterior three- dimensional coordinates associated with the predicted frequency peak values.

[0233] When interpolation techniques are used in element 1804, as discussed above, element 1816 of rendering includes presenting an interior virtual object at a corresponding interior three- dimensional location coordinate associated with the particular type of the intangible attribute signal. The interior virtual objects, for example, includes arrows and / or an illustration of the particular type of the intangible attribute signal.

[0234] Rendering, in one aspect of the present teachings, includes rending an intermediate virtual object at an intermediate object location, in the enhanced three-dimensional spatialized environment. The intermediate object locations correspond to a location of the corresponding intermediate three-dimensional location coordinate associated with the intermediate frequency peak value.

[0235] As discussed above, an eyewear (e.g., AR / VR headset) may be coupled to a handheld controller (e.g., hand-held controller 1150 of Figure 11) and a virtual representation of the handheld controller (e.g., virtual hand-held controller 1650 of Figure 16) may be presented in the virtual space of the eyewear. However, it may not always be advantageous to permanently show the virtual representation of the hand-held controller. Therefore, the present teachings offer amethod of removing the virtual representation of the hand-held controller from the virtual space. To this end, in one implementation of the present teachings, method 1800 includes an obtaining element. The obtaining element includes obtaining, from one or more position sensors located on the AR / VR headset and a tracking module, three-dimensional coordinates of the hand-held controller. Another element includes removing, from the enhanced three-dimensional spatialized environment, the three-dimensional representation of the hand-held controller when the handheld controller exceeds a predetermined distance from the AR / VR headset.

[0236] The hand-held controller may also be used to control what is visually rendered in the enhanced three-dimensional spatialized environment, such as rendering one of the intangible attribute signals and intensity associated with the intangible attribute signal. To this end, method 1800 further may further include receiving, from a user using the hand-held controller, a selection signal indicating selection by the user of a controller of a first type associated with an intangible attribute signal, which is one of the intangible attribute signals. Element 1816 of rendering the enhanced three-dimensional spatialized environment includes rendering, within the enhanced three-dimensional spatialized environment, the selected intangible attribute signal and the controller of the first type in an engaging manner with the virtual representation of the handheld controller. By way of example, the virtual representation of the hand-held controller may show a button (i.e., controller of a first type) being pressed.

[0237] To render intensity associated with the intangible attribute signal, method 1800 further includes an element of receiving, from the user using the hand-held controller, an intensity selection signal of a controller of a second type associated with the selected intangible attribute signal. Another element includes obtaining a numerical value of the intensity of the selected intangible attribute signal. Element 1816 of rendering the enhanced three-dimensional spatialized environment includes rendering a numerical value of the intensity value of the selected intangible attribute signal and a visual representation of the controller of the second type in an engaging manner with the virtual representation of the hand-held controller. By way of example, the virtual representation of the hand-held controller may show a slider (i.e., controller of a second type) being moved laterally a first end to a second end.

[0238] In one embodiment of the present arrangements, the processor may be configured to modify an image and / or a video, which may be a virtual image and / or virtual video, to include at least one visual aspect selected from a group including (i) the intangible attribute signal (or alocation thereof) providing the radio signal at the selected frequency, (ii) one or more virtual indicators towards such location and / or a direction of such location, (iii) one or more virtual indicators away from such location and / or the direction of such location, (iv) indicate a path towards, or away from, such location and / or the direction of such location, (v) highlight or dampen a surface proximate to such location and / or the direction of such location, and (vi) remove or hide a surface (such as an object) proximate to such location and / or the direction of such location. In some examples, the image and / or the video may pertain to a computergenerated environment ( / ., a virtual space). For example, the processor may be configured to trigger a display device, in communication with the eyewear, to display the modified image (and / or the modified video). In some instances, the processor may operate the display device to display an image (and / or video) both with or without the above modification. In further instances, the processor may be configured to initiate or inhibit the computer-generated environment (i .e., a virtual space) in response to (1) such location of the intangible attribute signal providing the radio signal at the selected frequency in a predetermined frequency block, (2) the direction of such location, and / or (3) the modified image and / or video. Other instances may include a processor may be configured to operate, or cease to operate, a remote device in response to the modified image and / or video.

[0239] The present systems and methods are not limited to the location of an energy source and may be used for detection of a gas leak. In connection with a gas leak, the present teachings may include the processor being configured to identify flow patterns (ambient inside a medium) emanating from a source of the gas leak.

[0240] In addition of the exemplar methods describe above, other examples of identifying the location of an object in three-dimensional space known in the art may also be contemplated for identifying an attribute source. By way of example, direct georeferencing, photogrammetric triangulation, single view metrology, ray casting with terrain model intersection, stereo vision, and structure from motion methodologies may be implemented, alone or in combination, to identify the intangible attribute signal in the real space.

[0241] Method 1800 may be described in the general context of computer executable instructions. Generally, computer executable instructions may include routines, programs, objects, components, data structures, procedures, modules, functions, and like that perform particular functions or implement particular abstract data types. The computer executableinstructions may be stored on a computer readable medium and installed or embedded in an appropriate device for execution. The order in which method 1800 is described is not intended to be construed as a limitation, and any number of the described method blocks may be combined or otherwise performed in any order to implement method 1800, or an alternate method. Additionally, individual elements may be deleted from method 1800 without departing from the concepts described herein. Furthermore, method 1800 may be implemented in any suitable hardware, software, firmware, or combination thereof, that exists in the art, related art, or that is later developed. Method 1800 describes an exemplary implementation eyewear or components thereof (e.g., processing subsystem 413 of Figure 4). One having ordinary skill in the art will understand that method 1800 may be modified appropriately for implementation with other configurations and methods or by any components of the eyewear or remote devices without departing from the concepts described in the present application.

[0242] The present teachings also provide methods for rendering a gas leakage. The method 1900 of Figure 19, according to one embodiment of the present teachings, preferably, begins with element 1902 including obtaining a first type of dataset, using a first sensor disposed on an AR / VR headset and that measures a first attribute at one or more three-dimensional coordinates defining a region or location in real space. The first type of dataset includes a first attribute value and an associated the three-dimensional coordinates where the first attribute value is obtained. In one embodiment of the present teachings, a ground positioning component (e.g., position sensor 435 of Figure 4) and an imaging sensor (e.g., optical sensor device 405 of Figure 4) provide or facilitate provision of the image value dataset. The ground positioning component informs on the position of the measurements taken when obtaining the image value dataset. The ground positioning component, in preferred embodiments of the present arrangements, uses relative positioning techniques, as opposed to making an absolute position measurement, as encountered when using a global positioning system (“GPS”) sensor, to arrive at the three-dimensional coordinates. Further, the three-dimensional coordinates, in these preferred embodiments, are obtained using positioning measurements relative to one or more reference points or coordinates. The ground positioning component may be disposed in eyewear (e.g., eyewear 222 of Figure 2 or eyewear 322 of Figure 3).

[0243] The imaging sensor, like the ground positioning component, be disposed in an eyewear. If the present arrangements rely on obtaining the image value dataset from a storage medium,such as an image value database, then the storage medium may reside inside or be communicatively coupled to the eyewear so that the image value dataset information is available for processing as described below.

[0244] Element 1902 of obtaining is substantially similar to the obtaining element 1802 of Figure 18 and the various embodiments described in relation to element 1802 may similarly be implemented in this element.

[0245] Method 1900 further includes element 1904, which involves obtaining a second type of dataset, using a second sensor that couples to the AR / VR headset that operates within a range of frequency that includes one or more frequency blocks. The second sensor measures for at least one of the frequency blocks, intensity values at one or more of the three-dimensional coordinates in the real space. A point of origin of the second sensor is at an offset distance relative to the point of origin of the first sensor. The second sensor measures, in one embodiment of the present teachings, a frequency that ranges from between about 3 terahertz and about 750 terahertz. In another embodiment of the present teachings, the second sensor measures a frequency that ranges from between about 119 terahertz and about 428 terahertz. Element 1904 of obtaining is substantially similar to the obtaining element 1804 of Figure 18 and the various embodiments described in relation to element 1804 may similarly be implemented in this element.

[0246] Following element 1904, element 1906 is carried out, which includes performing, for the frequency block, a fast Fourier transformation on the intensity values to produce a frequency domain dataset. The frequency domain dataset includes one or more frequency peak values and three-dimensional coordinates of the frequency peak values. Element 1906 of performing is substantially similar to the performing element 1806 of Figure 18 and the various embodiments described in relation to element 1806 may similarly be implemented in this element.

[0247] After element 1906 is carried out, element 1908 is performed, which includes identifying, based on one or more the frequency peak values and without using the first type of dataset, presence of one or more gas signals associated with one or more gases inside the real space. The performing element 1908 substantially similar to the identifying element 1808 of Figure 18 and the various embodiments described in relation to element 1808 may similarly be implemented in this element.

[0248] In on implementation of the present teachings, a user, user implementing method 1900, may desire to know that a gas leakage is occurring in the reals space but not the particular type ofgas that is leaking. By way of example, the user may already know what gases may be present in an area of interest (e.g., a pipe transmitting a known gas type) and is, therefore, only concerned about the presence of gas leaking from the pipe. To identify presence of gas, but not the gas type element 1908, in embodiment of the present arrangements, includes identifying one or more of the frequency peak values that are equal to or above a predetermined gas detection threshold value assigned to each of the frequency blocks. Frequency peak values that are equal to or above a predetermined gas detection threshold value are identified as a gas signals associated with one or more gases that are present in the real space. Frequency peak values that are below the predetermined gas detection threshold value may be identified as noise and are discarded.

[0249] In another implementation of the present teachings, a user may desire to know the type of gases that are leaking. However, as discussed above, the present teachings recognize that identifying some intangible attribute signals, such as gas signal associated with a gas, by a single frequency peak may not be feasible. Multiple gas signal associated with different gases, for example, may have the same or similar frequency peak value or exist within a frequency block. Therefore, multiple frequency peaks may be used to identify one or more different intangible attribute signals inside the real space.

[0250] The presence of a particular gas, in one aspect, is identified using a predefined spatial reference pattern and / or a temporal reference pattern that is associated with the particular gas. Element 1906 of performing, on one embodiment of the present teachings, includes producing a frequency domain dataset that includes multiple temporal frequency peaks, each having a temporal frequency peak value, and / or multiple spatial frequency peaks, each having a spatial frequency peak value. Element 1908 of identifying includes matching, within a predefined tolerance, the multiple spatial frequency peaks with the predefined spatial reference pattern and / or the multiple temporal frequency peaks with the temporal reference pattern to identify presence in the real space of one or more types of the gas.

[0251] The spatial reference pattern, in one implementation of the present teachings, includes two or more reference spatial frequency peaks, each of the reference spatial peaks having a reference spatial frequency value and the temporal reference pattern includes two or more reference temporal peaks, each of the reference temporal peaks having a reference temporal frequency value. Matching includes matching, within the predefined tolerance, two or more of the multiple spatial frequency peak values with two or more of the reference spatial frequencyvalues associated with the spatial reference pattern and / or two or more of the multiple temporal frequency peaks values with two or more of the reference temporal frequency values associated with the temporal reference pattern.

[0252] Following the identifying of element 1908, element 1910 is performed, which includes spatializing, using a plurality of the three-dimensional coordinates, the first type of dataset and the offset distance to create a first type of spatialized dataset and a spatialized offset distance. Spatializing is preferably implemented using a three-dimensional spatializing model. In preferred embodiments of the present teachings, an image spatializing module (e.g., image spatializing module 576 of Figure 5 A or image spatializing module 576' of Figure 5B) is used to create the spatialized image value dataset. The image spatializing module may reside on the eyewear or on an external processor. If the image spatializing module resides on the external processor, then the external processor, which is external to the eyewear, is communicatively coupled to the eyewear to convey to it the “processed information,” resulting from processing of the spatialized image value dataset.

[0253] In those embodiments where an image spatializing module is used in element 1910, this module begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the image spatializing module integrates the subdivisions to create a spatialized model of the real space. In the spatializing element 1910, the image values may be distributed, based upon the spatialized model, to create the spatialized image value dataset. Further, “real space,” in the present methods, is electronically represented using a plurality of the three-dimensional location coordinates, which may have been retrieved from obtaining elements 1902 and / or 1904. This spatializing element 1910 is substantially similar to the determining element 1810 of Figure 18 and the various embodiments described in relation to element 1810 may similarly be implemented in this element.

[0254] Method 1900 further includes an element 1912, which includes spatializing, using the plurality of the three-dimensional coordinates, the second type of dataset to create a second type of spatialized dataset. An attribute spatializing module, which may reside on the eyewear or on an external processor, may perform element 1912. If the attribute spatializing module resides on the external processor, then the external processor, which is external to the eyewear, is communicatively coupled to the eyewear to convey to it the “processed information,” resulting from processing of the spatialized frequency domain dataset.

[0255] In those embodiments where an attribute spatializing module is used in element 1912, this module, like the image spatializing module of element 1910, begins the spatializing element by, preferably, spatially partitioning the real space into plurality of subdivisions. Then, the attribute spatializing module integrates the subdivisions to create a spatialized model of the real space. In spatializing element 1912, one or more of the frequency peak values may be distributed, based upon the spatialized model, to create one or more spatialized frequency peak values.

[0256] In one implementation of method 1900, if a spatialized model is created using an image spatializing module or an attribute spatializing module, then the other spatializing module is free to use the created spatializing model, without the need for creating another or recreating a spatialized model to effect distribution of the data within the first type of dataset or second type of dataset, as the case may be. Element 1912 of spatializing is substantially similar to the obtaining element 1804 of Figure 18 and the various embodiments described in relation to element 1804 may similarly be implemented in this element.

[0257] Next, an element 1914 is carried out, which involves aligning, using the spatialized offset distance, the first type of spatialized dataset with second type of spatialized dataset to create an enhanced three-dimensional spatialized environment. This aligning element 1914 is substantially similar to the determining element 1814 of Figure 18 and the various embodiments described in relation to element 1814 may similarly be implemented in this element. The aligning module may reside on an eyewear or on an external processor, which is disposed external to and communicatively coupled to the eyewear.

[0258] Regardless of where it is located, the aligning module may use one or more common spatial features or common cartesian coordinates present in both the spatialized frequency domain dataset and the spatialized image value dataset to effect alignment of the two spatialized datasets. Examples of such common spatial features include vertex, longitude, or latitude. Using one or more common vertices, found in both the spatialized frequency domain dataset and the spatialized image value dataset, to effect alignment represents an embodiment of element 1914.

[0259] After completing the alignment of two spatialized datasets, method 1900 proceeds to element 1916, which includes rendering, in a virtual space in the AR / VR headset and using a rendering engine, the enhanced three-dimensional spatialized environment, indicating, in the enhanced three-dimensional spatialized environment, presence and concentration of one or morethe gases. In one implementation, different types of gases are rendered in different colors of a color gradient.

[0260] Element 1916 of rendering, in one implementation of the present teachings, includes rendering one of the gases such that a first concentration range of the gas is represented by a first color (e.g., red), a second concentration range of the gas is represented by a second color (e.g., yellow), and a third concentration range of the gas is represented by a third color (e.g., green), wherein the second concentration range is of a lower magnitude than the first concentration range and the third concentration range is lower than the first concentration range and the second concentration range. Figure 17 shows an exemplar rending of a gas leaking from a pipe, where the leaking gas is visually represented using a first concentration range 1708, a second concentration range 1706, and a third concentration range 1704.

[0261] A rendering element e.g., rendering engine 582 of Figure 5A or rendering engine 582’ of Figure 5B) may render the enhanced three-dimensional spatialized environment on a display component (e.g., VO 417 of Figure 4). In certain exemplary embodiments, the display component is integrated into a client device, which is at least one device chosen from a group comprising wearable, smartphone, and computer. In preferred embodiments of the present teachings, the wearable is a laptop. The rendering element substantially similar to element 1816 of Figure 18 and the various embodiments described in relation to element 1816 may similarly be implemented in this element. Figures 16 and 17 show exemplary images produced in virtual space from element 1916.

[0262] The present teachings offer novel methods for determining, based on the frequency domain dataset and without using the imaging, one or more locations and / or directions of locations of one or more of the identified intangible attribute signals. These novel methods may be incorporated into method 1800 or 1900.

[0263] In one embodiment of the present teachings, method includes performing a clustering analysis one or more locations and / or directions of locations of one or more of the identified intangible attribute signals. The goal of a clustering analysis is to determine the location and / or the direction of location of one or more of these intangible attribute signals. Clustering analysis may segregate intangible attribute measurements with similar traits into clusters. In one embodiment of the present teachings, clustering analysis analyzes the distance between one or more of the X-, Y-, and Z-coordinate values of two different clusters or the lapse of timebetween instances in time of detection clusters of certain frequency regimes to provide a direction of propagation in a three-dimensional space, of one or more frequencies generated by one or more intangible attribute signals. When the non-imaging sensor detects and measures this range of frequencies, and, preferably in real time, generates a responsive signal, the present teachings are able to offer the processor configured to compute a direction of propagation of a signal, which corresponds to the direction of the propagation of this range of frequencies and associate the direction of propagation of the signal to the particular intangible attribute signal that generated this range of frequencies.

[0264] In another embodiment of the present teachings, k-means clustering analysis is used to reveal the location and / or the direction of location of each intangible attribute signal. While not wishing to be bound by theory, the present teachings recognize that the processor may implement a method of Zr-means clustering to determine a centroid for a predetermined number of clusters, k, wherein each centroid of each cluster is the mean of all data points within the cluster. Each cluster may include one or more data points such as signal strengths, which may correspond to (i) various radio signals and / or (ii) various frequencies in the radio signal, such as the radio signal 1802, received by the non-imaging sensors on the eyewear 322. The number of clusters, is the predetermined number of clusters that will be created during the analysis. The number of predefined clusters, in one embodiment of the present teachings, ranges from between about 2 clusters and about 10 clusters. In a preferred embodiment of the present teachings, the number of predefined clusters ranges from between about 4 clusters and about 8 clusters. In a preferred embodiment of the present teachings, the number of predefined clusters ranges from between about 5 clusters and about 7 clusters.

[0265] Determining the location and / or the direction of location of an intangible attribute signal using k-means clustering, in one embodiment of the present teachings, includes the processor being configured for identifying centroids of various clusters. Each cluster may include a centroid, which may correspond to the mean signal strength in that cluster. At each position ( / .< ., X-, Y-, Z-coordinate values) of the eyewear 322 in the real space, the processor may compare a mean signal strength (or centroid) of each cluster with a signal strength (e.g., a temporal frequency peak having the highest temporal frequency strength value) of the radio signal (or a portion thereof) at a frequency (e.g., a temporal frequency value associated with the temporal frequency peak having the highest temporal frequency strength value). Based on the comparison,when there is a match, or a substantial match (e.g., within + / - 10% of the signal strength value), between the mean signal strength and the signal strength, the centroid may indicate a location of the intangible attribute signal providing the radio signal (or a portion thereof) at the frequency.

[0266] Further, as time lapses from T1 to T2 and then to T3, during different instances in time of detection of certain frequency regimes, the clustering analysis of the present teachings may include the processor being configured to incrementally analyze the signal strength measurements being obtained and compare them with the signal strength corresponding to a selected frequency (e.g., a temporal frequency peak value having the highest temporal frequency strength value).

[0267] Another novel method for determining, based on the frequency domain dataset and without using the imaging, one or more locations and / or directions of locations of one or more of the identified intangible attribute signals is described. A processor, on one embodiment of the present teachings will perform elements described below for each of the intangible attribute sources, whether received simultaneously or at different instants by the non-imaging sensors. However, the description of these elements is explained herein by way of an exemplary radio signal 1602 of Figure 16 for the sake of brevity and simplicity.

[0268] A first element includes which includes calculating, for each of the intangible attribute sources, a phase angle at a plurality of one or more non-imaging sensors. The plurality of nonimaging sensors includes a first non-imaging sensor and a second non-imaging sensor having a predetermined baseline distance there between. In one embodiment of the present teachings, a non-imaging sensor device (e.g., second sensor 440 of Figure 4) includes the first non-imaging sensor and the second non-imaging sensor (hereinafter collectively referred to as “non-imaging sensors”) for receiving the radio signals.

[0269] In one embodiment, the processor may be configured to determine an amplitude and a difference in phase of each of the radio signals received at each of the non-imaging sensors. The “phase” and “phase angle” are used interchangeably in the context of their broadest definitions. The “phase angle” may refer to an angular component of a signal’s waveform at an input of a receiver. The phase angle may indicate a position of the signal’s waveform along its oscillatory cycle at the moment it is received by a sensor such as the non-imaging sensor. The phase angle may represent the relative position of the waveform at a specific point in time compared to a first reference point. Examples of the first reference point may include a start of the waveform, a peakamplitude, or a zero-crossing point of the radio signal. For example, the non-imaging sensors may receive the radio signal (in time domain) as shown in Equation 1. y(t) = A sin sin (a>t + 0) (1) where: y(t) = sinusoidal signal (e.g., radio signal)A = amplitude of the sinusoidal signal a> = angular frequency t = time0 = phase angle

[0270] Based on Equation 1, in one example, if a sinusoidal portion of the radio signal is received at its peak with the zero-crossing point (representing zero amplitude) as the first reference point, the processor may determine the phase angle to be 7t / 2 radians (i.e., 90 degrees). This means that the signal waveform is at its maximum positive amplitude at the moment the radio signal may be received at the non-imaging sensors. As such, the processor may measure a first phase angle 01, e.g., at the first non-imaging sensor, and a second phase angle 02, e.g., at the second non-imaging sensor.

[0271] Another element is performed on each intangible attribute received from an intangible attribute source. This element includes determining, based on the phase angle of each intangible attribute at each of the plurality of non-imaging sensors, a phase difference of each of the intangible attribute sources.

[0272] A calculation of a phase difference is represented in Equation 2.A0 = 01 - 02 (2)

[0273] Following the determining element the method proceeds to an element of selecting, which includes selecting, for each of the intangible attribute sources, one of one or more spatial frequency values and / or one of one or more temporal frequency values. In one embodiment of the present teachings, the selected temporal frequency value and / or spatial frequency value (hereinafter also referred to as “the selected frequency”) is the highest temporal frequencystrength value and / or highest spatial frequency strength value in a temporal frequency pattern and / or spatial frequency pattern. The selected temporal frequency value and / or spatial frequency value, in another embodiment of the present teachings, is the highest temporal frequency strength value and / or highest spatial frequency strength value in a temporal frequency band and / or spatial frequency band.

[0274] Moreover, where multiple intangible attribute sources are identified in a real space, each of which generates a temporal strength value at a particular temporal frequency, the highest temporal frequency strength value may indicate that an intangible attribute source providing the corresponding radio signal (or a portion thereof) is spatially closest to the AR / VR headset such as the eyewear 322 (or the non-imaging sensors) compared to any other intangible attribute sources in the real space.

[0275] Moreover, where multiple intangible attribute sources are identified in the real space, each of which generates a temporal strength value at different temporal frequencies, the highest temporal frequency strength value may indicate that an intangible attribute source providing the corresponding radio signal (or a portion thereof), is spatially closest to the AR / VR compared to any other intangible attribute sources in the real space.

[0276] The particular frequency band (e.g., 1000 megahertz - 1500 megahertz) to which the selected frequency may belong, may be selected or predefined based on a predetermined target frequency of the intangible attribute source. For example, a temporal peak and / or spatial peak may be deemed relevant and thus selected by the processor when the related temporal frequency value and / or spatial frequency value is within the predetermined or preselected frequency band of interest. In one embodiment of the present teachings, the processor is preconfigured or dynamically configured to select one or more higher frequency bands (e.g., those equal to or greater than 1000 megahertz) to identify low voltage sources. In another embodiment of the present teachings, the processor is preconfigured or dynamically configured to select one or more lower frequency bands e.g., those less than 1000 megahertz) to identify relatively high voltage sources, in the real space proximate to the AR / VR headset such as the eyewear 322 (or the nonimaging sensors). The selection of the frequency band of interest and / or the peak frequency therein may assist the processor in differentiating between various intangible attribute sources that may emit radio signals at different frequencies.

[0277] Next, an element includes determining, based on the selected frequency, an instantaneous wavelength of each intangible attribute source. The instantaneous wavelength of the intangible attribute (or a portion thereof) at the selected frequency may be determined using Equation 3. = -f(3) where:A = relative wavelength (or instantaneous wavelength) of the intangible attribute (e.g., radio signal)C = speed of light f = instantaneous frequency (or the selected frequency) of the intangible attribute

[0278] Following the element of determining, an element includes calculating, for each of the intangible attribute sources, based on the baseline distance, the phase difference, and the instantaneous wavelength, an angle of arrival at each of the plurality non-imaging sensors. The angle of arrival indicates a direction in which the intangible attribute source is located.

[0279] In one embodiment of the present teachings, the processor (or the non-imaging sensors) may be configured to calculate an angle of arrival of an intangible attribute (e.g., radio signal) emitted by an intangible attribute source, using Equation 4.where:A< > = phase difference between the intangible attribute received at each of the non-imaging sensors d = baseline distance between the non-imaging sensorsA = instantaneous wavelength of the radio signal at the selected frequency01= angle of arrival of the intangible attribute at one of the non-imaging sensors (e.g., first nonimaging sensor)

[0280] In Equation 4, the angle of arrival, 01, may account for the directional sensitivity of the corresponding sensor, such as the first non-imaging sensor. The 0rindicates that the angle ofarrival may be determined relative to a normal to a straight line (e.g., baseline distance) joining the first non-imaging sensor and the second non-imaging sensor. The angle of arrival, 01, may indicate a direction in which an intangible attribute source may be located relative to the first non-imaging sensor, where such intangible attribute source is providing the radio signal (or a portion thereof) at the selected frequency at a given instant. Further, the processor may be configured to calculate a second angle of arrival of the radio signal at the second non-imaging sensor, using Equations 5 and 6.A = Z2LA. sinsin (t— 02) (5)where:A< > = phase difference between the radio signal received at each of the non-imaging sensors d = baseline distance between the non-imaging sensorsA = instantaneous wavelength of the radio signal at the selected frequency02= angle of arrival of the radio signal at the other non-imaging sensor (e.g., second nonimaging sensor)

[0281] In Equation 6, the angle of arrival, 02, may account for the directional sensitivity of the corresponding sensor, such as the second non-imaging sensor. The angle of arrival, 02, may be determined relative to a normal to a straight line (e.g., baseline distance) joining the first nonimaging sensor and the second non-imaging sensor. The angle of arrival, 02, may indicate a direction in which an intangible attribute source may be located relative to the second nonimaging sensor, where such intangible attribute source is providing the radio signal (or a portion thereof) at the selected frequency (e.g.., selected frequency of the selected radio signal such as radio signal at a given instant).

[0282] Next, a determining element is carried out, which includes determining, for each of the intangible attribute sources, based on the angle of arrival at each of the plurality of non-imaging sensors and the baseline distance, a spatial location of each intangible attribute source from oneof the AR / VR headsets. The spatial location is determined relative to a current position of one of one or more of the AR / VR heads in real space.

[0283] The processor, in one example, may triangulate the spatial location of the intangible attribute source using suitable triangulation algorithms known in the art. For example, the processor may determine a spatial distance (SD) from the eyewear 322 to the intangible attribute source, using Equation 7.where:SD = spatial distance to the intangible attribute source providing the radio signal (or portion thereof) at the selected frequency d = baseline distance between the first non-imaging sensor and the second non-imaging sensor = angle of arrival of the radio signal at the first non-imaging sensor02= angle of arrival of the radio signal at the second non-imaging sensor

[0284] In Equation 7, the spatial distance may refer to a normal (e.g., perpendicular distance) from the baseline distance to the intangible attribute source providing the radio signal at the selected frequency. The spatial distance may assist in determining the spatial location of the intangible attribute source with respect to the eyewear 322 (or the non-imaging sensors). For example, the processor may determine the (x, y) coordinates of the intangible attribute source, using Equations 8 and 9. x sin sin + y cos cos , = d (8) x sin sin2+ y cos cos2= d (9) where: d = baseline distance between the first non-imaging sensor and the second non-imaging sensor 61= angle of arrival of the radio signal at the first non-imaging sensor92= angle of arrival of the radio signal at the second non-imaging sensor

[0285] Equation 8 may represent an equation of a plane containing the first non-imaging sensor and the intangible attribute source providing the radio signal at the selected. Similarly, Equation 9 may represent an equation of a plane containing the second non-imaging sensor and the intangible attribute source providing the radio signal at the selected temporal frequency value and / or spatial frequency value. Since the values of d, 0 , and 02maY be determined, as discussed above, the processor may solve Equations 8 and 9 to determine the values of x and y corresponding to (x, y) coordinates of the intangible attribute source in the real space. Further, the processor may be configured to estimate an elevation / depression from the AR / VR headset such as the eyewear 322 to the intangible attribute source providing the radio signal at the selected frequency for determining the z coordinate of the intangible attribute source in the real space. In one embodiment, the processor may be configured to determine (1) a vertical distance (or elevation) of each of the non-imaging sensors from the ground and (2) a distance to the intangible attribute source from each of the non-imaging sensors. The processor may determine vertical distances zl, z2 of the first and the second non-imaging sensors respectively using any suitable devices such as altimeters and / or rangefinders known in the art. In some examples, these altimeters and / or rangefinders may be mounted on the AR / VR headset such as the eyewear 322. Further, the processor may determine distances to the intangible attribute source from each of the non-imaging sensors, using Equations 10 and 11.(10)(H) where: d = baseline distance between the first non-imaging sensor and the second non-imaging sensorDI = spatial distance from the first non-imaging sensor to the intangible attribute sourceD2 = spatial distance from the second non-imaging sensor to the intangible attribute source9}= angle of arrival of the radio signal at the first non-imaging sensor02=angle of arrival of the radio signal at the second non-imaging sensor

[0286] Based on Equations 10 and 11, the processor may determine a vertical distance between the intangible attribute source and each of the non-imaging sensors, using Equations 12 and 13. vl = DI — zl (12) v2 = D2 — z2 (13) where: vl = vertical distance between the first non-imaging sensor and the intangible attribute source; v2 = vertical distance between the second non-imaging sensor and the intangible attribute source;DI = spatial distance from the first non-imaging sensor to the intangible attribute source;D2 = spatial distance from the second non-imaging sensor to the intangible attribute source; zl = vertical distance of the first non-imaging sensor from the ground; and z2 = vertical distance of the second non-imaging sensor from the ground.

[0287] Based on Equations 12 and 13, the processor may determine the elevation (or z- coordinate in the real space) of the intangible attribute source relative to the non-imaging sensors, using Equation 14. zrel= 2 — vl (14) where: zre(= relative z-coordinate of the intangible attribute source in the real space

[0288] Hence, the processor may determine the location of the intangible attribute source defined by (x, y, z) coordinates in the real space based on Equations 8, 9, and 14.

[0289] The present teachings offer novel methods for spatializing an image dataset. In one embodiment, spatializing begins with an element that includes obtaining boundary data of a real space. In certain embodiments, this element of the present teachings includes obtaining three- dimensional coordinates that define, z.c., electronically represent, a boundary of a scene. A scene is commonly present in real space in front of a user of a user device (e.g., a laptop).

[0290] Next, this method includes a subdividing element that requires subdividing the boundary data of the real space into a plurality of subdivisions. In certain embodiments, the subdividing element of the present teachings involves subdividing the electronically represented boundary obtained previously such that each of the resulting subdivisions is electronically represented by three-dimensional subdivision boundary coordinates. In other embodiments of the present teachings, the subdividing element includes spatially partitioning a real space into a plurality of subdivisions. In these embodiments, a plurality of corresponding three-dimensional location coordinates collectively forms an electronic representation of the real space, and this space undergoes spatial partitioning. The word “corresponding,” used in connection with the term “location coordinate,” conveys that the “location coordinate” is one where at least one pixelated or at least one voxelated value was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the pixelated or voxelated values obtained there.

[0291] This spatializing method then proceeds to obtaining three-dimensional pixelated data values or three-dimensional voxelated data values for corresponding location coordinates. In this element, each of the corresponding location coordinates is “associated” with at least one of the associated three-dimensional pixelated data values or at least one associated three-dimensional voxelated data values. The word “associated,” used in connection with the terms “three- dimensional pixelated data values” and “three-dimensional voxelated data values,” conveys that the pixelated or the voxelated values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0292] In one example of this implementation, obtaining the three-dimensional pixelated data values or the three-dimensional voxelated data values includes measuring light intensity or measuring color intensity present at a unit pixel area or a unit voxel (volumetric) space, respectively, of an imaging sensor device (e.g., imaging sensor 415 of Figure 4).

[0293] According to one embodiment of the present teachings, using an optical sensor or a database to obtain the required data values may be similarly implemented.

[0294] Once the subdivisions are obtained, the spatializing method of the present teachings may proceed to an element that involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and this element generates one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisions that include at least onethree-dimensional location coordinate, i.e., at least one location from where a measurement of the pixelated or the voxelated values was obtained.

[0295] Then, the spatializing method of the present teachings advances to an assigning element that includes assigning at least one of the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values to one or more of the selected subdivisions to define one or more assigned subdivisions. In this element, each of the three- dimensional pixelated data values or the associated three-dimensional voxelated data values, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated three-dimensional pixelated data values or at least one of the associated three-dimensional voxelated data values.

[0296] In one preferred aspect, the assigning element includes assigning at least one of the associated three-dimensional pixelated data values or at least one of the associated three- dimensional voxelated data values to an entire portion of the selected subdivisions. In another preferred aspect, where two or more pixelated or voxelated values are available, the assigning element includes assigning a summation of weighted values of two or more of the associated three-dimensional pixelated data values or a summation of weighted values of two or more of the associated three-dimensional voxelated data values to the selected subdivisions.

[0297] After the conclusion of the assigning element, the spatializing method of the present teachings preferably proceeds to an integrating element, which involves integrating the assigned subdivisions to form a spatialized image dataset. In certain preferred embodiments, the integrating element of the present teachings includes integrating a plurality of the three- dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized image dataset. In other preferred embodiments, the integrating element of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values are distributed, based on the spatialized model, to create the spatialized image dataset.

[0298] Regardless of how they are formed, the spatialized image data set of the present teachings may be further used in other methods described herein to achieve other novel results and thespatialized image data set obtained in method 1900 represents the “processed information” that is rendered in a subsequent element on a display component of a user device, eyewear, or smartphone.

[0299] To this end, in an optional implementation, the spatializing method of the present teaching is subjected to rendering, which uses a rendering engine similar to the one described in connection with method 1900, to render the spatialized image dataset on a display component of a client device.

[0300] In another aspect, the present teachings offer alternate novel methods for spatializing, using a spatializing model, a second type of dataset. This alternate method of spatializing may begin with an obtaining element that includes obtaining boundary data of a real space. In certain embodiments of the obtaining element, three-dimensional coordinates that define, z.c., electronically represent, a boundary of a scene is obtained.

[0301] Next, the alternate method of spatializing includes a subdividing element that requires subdividing the boundary data of the real space into a plurality of subdivisions. By way of example, the subdividing element may involve subdividing the electronically represented boundary obtained in the obtaining element such that each of the resulting subdivisions is electronically represented by three-dimensional subdivision boundary coordinates. As another example, the subdividing element may include spatially partitioning the real space into a plurality of subdivisions. In this example, a plurality of corresponding three-dimensional location coordinates collectively forms an electronic representation of the real space that undergoes spatial partitioning. The word “corresponding,” used in connection with the term “location coordinate,” conveys that the “location coordinate” is one where at least one attribute value of a particular type was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the attribute value obtained there.

[0302] The alternate method of spatializing also includes another obtaining element, which requires obtaining one or more different types of attribute values for corresponding location coordinates. In this element, each of the corresponding location coordinates is “associated” with at least one type of attribute value. The word “associated,” used in connection with the terms “attribute values,” conveys that one or more attribute values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0303] In one example of this obtaining element (i.e., another obtaining element), a magnitude of an intangible property of a real space is obtained. In one implementation of this example, this obtaining element includes detecting, inside the real space, a value of a parameter, e.g., magnitude, of at least one type of attribute that is chosen from a group comprising throughput of a connectivity signal, latency of a connectivity signal, interference of a connectivity signal, volatility of a connectivity signal, stability of a connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature.

[0304] According to one embodiment of the present teachings, this obtaining element is carried out using a non-imaging sensor or a database to obtain the required data values.

[0305] Once the subdivisions are obtained, the alternate method of spatializing may proceed to an identifying element, which involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and the identifying element produces one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisions that include at least one three-dimensional location coordinate, i.e., at least one location from where a measurement of at least one attribute value of a particular type was obtained.

[0306] Then, the alternate method of spatializing advances to an assigning element that includes assigning at least one of the associated attribute values of at least one type to one or more of the selected subdivisions to define one or more assigned subdivisions. In this element, each of the associated attribute values of a particular type, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated attribute values of a particular type.

[0307] In one preferred aspect, the assigning element includes assigning at least one of the associated attribute values of a particular type to an entire portion of the selected subdivisions. In another preferred aspect, where two or more attribute values of a particular type are available, the assigning element includes assigning a summation of weighted values of two or more of the associated attribute values of a particular type to the selected subdivisions.

[0308] After the conclusion of the assigning element, the alternate method of spatializing preferably proceeds to an integrating element, which involves integrating the assigned subdivisions to form a spatialized second type of dataset. In certain preferred embodiments, the integrating element of the present teachings includes integrating a plurality of the three- dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized second type of dataset. In other preferred embodiments, the integrating element of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated attribute values of at least one type are distributed, based upon the spatialized model, to create the spatialized second type of dataset.

[0309] Regardless of how they are formed, the spatialized attribute value data set of the present teachings may be further used in other methods described herein to achieve other novel results and the spatialized attribute value data set represents the “processed information” of the alternate method of spatializing and is subsequently rendered on a display component of, a user device, eyewear, or smartphone.

[0310] To this end, in an optional implementation, the alternate method of spatializing may advance to a rendering element, which uses a rendering engine, to render the spatialized second type of dataset on a display component of a client device.

[0311] In yet another aspect, the present teachings offer methods for rendering an image dataset and one or more attribute values datasets.

[0312] In a preferred embodiment, this rendering method begins with an obtaining element, which includes obtaining three-dimensional pixelated data values or three-dimensional voxelated data values for one or more corresponding three-dimensional location coordinates. In this obtaining element, each of the corresponding three-dimensional location coordinates is associated with at least one associated three-dimensional pixelated data value or associated three- dimensional voxelated data value. This obtaining element is, preferably, carried out using an optical sensor and a ground positioning component.

[0313] Regardless of the methodology and components used, the rendering element provides an image of a real space (e.g., the interior and exterior of a tent). In certain embodiments of the present teachings, when such an image is taken, using an XR system of the present arrangements, metadata fields may be attached to it. These metadata fields may include at least one informationchosen from a group comprising model of an imaging device, time an image was taken, whether a flash was used during image capturing, shutter speed, focal length, light value, and a location information provided by a ground positioning component. Accordingly, collections of images may be used to map out a real space such as an office environment. As will be explained below, the data pertaining to the real space is processed, according to the present teachings, to visualize the real space in virtual reality or XR from many perspectives.

[0314] Among the many perspectives, a user or a viewer is not only allowed to visualize a three- dimensional replication of the real space, but one or more users or viewers traverse the space using conventional controls such as joysticks, three-dimensional controllers, and keyboards to gather the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values described in the obtaining element. To this end, preferably, multiple client devices and controllers under the possession and / or control or users or viewers gather, as is described above in the context of “edge computing,” the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values and the corresponding location coordinates described in the obtaining element.

[0315] The rendering method also includes another obtaining element that involves obtaining, using at least one non-imaging sensor, one or more different types of attribute values for one or more of the corresponding three-dimensional location coordinates. In this obtaining element, each of the corresponding three-dimensional location coordinates is associated with at least one type of associated attribute value.

[0316] The present teachings recognize that two different obtaining elements may include many novel implementation features. By way of example, in those instances when multiple client devices and / or external controllers, such as positioning controllers, may be employed to collect different types of data values (e.g., image data values or attribute values) from a user or a viewer, the client device or the external controller, using the ground positioning component or position sensors, disposed thereon, aware of its relative position with respect to one or more reference points or coordinates. Moreover, the client devices or the external controllers detect their position when it is within a viewable range on a mobile application and adjusts its own location or opacity or even extends virtual controls based on its location or based on its state. When one or more users or viewers move the client device or the external controller away from the reference point(s) or coordinate(s), the display might ultimately present the same image dataand / or the intangible property data for the real space, but the “processed information” may be rendered from a different viewpoint and / or may be rendered from a perspective that is further away in distance.

[0317] In the two different obtaining elements, and even certain subsequent elements, conventional three-dimensional mapping techniques may be employed to effectuate certain embodiments disclosed herein. By way of example, different types of datasets may employ different modeling technologies to optimize data visualization and three-dimensional interactivity. Moreover, certain dataset animations may be simulated by rapidly plotting datasets in succession.

[0318] As another example, graphical user interface elements may be anchored to a three- dimensional visualization or rendering that presents a state of the selected data analytics. For example, and without limitation, a slider could be mapped to an opacity of a data layer (z.e., corresponds to a dataset of a particular type). As the slider is moved up, the data layer threshold level of the particular type may change and the data layer becomes more opaque; conversely, as the slider is moved down, the data layer of the particular type becomes more transparent. These graphical elements may animate for better usability, such as fading in text or moving arrows and / or lines. In the slider layer example, the actual opacity value and / or a percentage may be drawn on the slider knob itself as it is being slid, and then disappear after a predetermined duration, e.g., about 2 seconds, from a time when the user had finished moving the slider. Thus, the graphical interface elements of the present teachings are capable of conveying both, a state of one or more different types of data, as well as, a state of a client device’s or an external controller’s buttons and relevant components.

[0319] The present teachings recognize that correlating image data with intangible property data allows XR systems of the present arrangements to present a multi-variable view of a physical space in a single rendering.

[0320] After the obtaining element, mentioned above, has obtained a requisite number of pixelated or voxelated values, the rendering method may proceed to a spatializing element, which requires spatializing the three-dimensional pixelated data values or the three-dimensional voxelated data values to create a spatialized image dataset. In preferred embodiments, the spatialized element relies upon a plurality of the corresponding three-dimensional locationcoordinates to arrive at the spatialized image dataset. By way of example, one or more elements described in the rendering method may be carried out to create the spatialized image dataset.

[0321] After the rendering method has obtained a requisite number of attribute values, it may proceed to a spatializing element, which requires spatializing the associated attribute values of at least one particular type to create a spatialized second type of dataset of at least that type. This spatializing element, like the above-mentioned spatializing element, may rely upon a plurality of the corresponding three-dimensional location coordinates to arrive at the spatialized second type of dataset.

[0322] After a spatialized image dataset and a spatialized second type of dataset are obtained, the rendering method proceeds to an aligning element, which includes aligning the image spatialized dataset with the attribute value spatialized dataset to create an enhanced three-dimensional spatialized environment. By way of example, one or more common features, e.g., vertex or vertices, longitude, or latitude, present in both the image spatialized dataset and spatialized second type of dataset, are used to effect alignment. The enhanced three-dimensional spatialized environment of the aligning element represents the “processed information” of the rendering method, and in a subsequent element is rendered on a display component of a user device, an eyewear, or a smartphone.

[0323] As a result, the rendering method then proceeds to a rendering element, which involves rendering, using a rendering engine and on a display component, the enhanced three-dimensional spatialized environment.

[0324] Each of method 1800 of Figure 18 and method 1900 of Figure 19 represent different types of programmable instruction of the present teachings that reside on XR systems of the present arrangements. The present teachings recognize that, although use of XR systems of the present arrangements represent a preferred mode of implementation, these programmable instructions may be implemented using other structural details not described herein.

[0325] In one embodiment, programmable instructions of the present teachings, rely on complex data spatialization, sampling algorithms, and machine learning. When the different present methods described herein are implemented as a software application, such applications discover signals emanating from nearby communication devices, which rely on radio waves for wireless connectivity, e.g., Wi-Fi™ routers and Bluetooth™ devices. This information is preferablydisplayed back in a fully interactive, completely immersive overlay shown through a display of a present XR system, like a headset or smartphone.

[0326] The present teachings recognize that a distinction may be made between a signal collection device and a signal visualization device, even though they may be the same device in certain embodiments of the present teachings. A signal collection device, commonly called a sensor, is classified as one possessing a measurement device. In the context of radio frequency data, this would be a device containing an RF receiver and antenna operating on protocols like Wi-Fi™, Bluetooth™, ZigBee™, Z-Wave™, 3G™, 4G™, LTE™ and 5G™. A signal visualization device, on the other hand, is typically characterized by a display capable of showing objects mapped into real space. Visualization may include the use of holographic elements to create the perception of objects floating in front of the user, as in the case of AR headsets or smartphones. VR headsets may use simulated environments that allow remote viewing of a real space. Mixed Reality (“MR”) and / or XR devices are typically a combination of both technologies, z.e., AR and VR, and allow for either type of experience.

[0327] Examples of devices that include both signal collection and signal visualization components include HoloLens™ available from Microsoft™, Magic Leap One™ available from Magic Leap™, and smartphones like iPhone X™ available from Apple™, Inc., and Galaxy S10™ available from Samsung™. Examples of devices that include only signal collection components include specialized RF antenna, microphone, and Geiger counter. Examples of devices that include only signal visualization components include Meta™ 2 (i.e., an AR headset), Oculus Rift™ (i.e., VR headset), and HTC Vive™ (i.e., MR headset).

[0328] In one embodiment of the present arrangements, signal measurements are accomplished by mounting many kinds of sensors to the signal collection device, and these sensors are discussed in connection with Figures 2 and 3 described herein.

[0329] The present teachings recognize that by using sensor fusion and implicit directionality from the eyewear, triangulation of signals is possible. Further, by using headset-mounted or standalone microphones, advanced audio profiles may be built of the real space that may model sources of noise as well as noise propagation characteristics like the reverberation factor of the materials. The present teachings further recognize that using special high-pass and low-pass filters allows triangulation of certain types of equipment, like low-humming industrial generators, or high-pitched electrical transformers. Even in a typical home, devices liketelevisions, computers, and wireless routers emit distinct patterns of noise that is beyond the human ear’s range but within the microphone’s range. Such recognition allows the present teachings to accomplish audio fingerprinting. Representative devices used for audio fingerprinting include at least one device chosen from a group comprising thermometer, hygrometer, barometer, and airflow sensor. In certain embodiments, the present teachings recognize that the use of simple climate instruments allows measurement of and ultimately visualization of signals in 3D to accomplish such things as HVAC mapping, microclimate analysis, chemical tracing, insulation planning, and food storage auditing.

[0330] The sensors used in the present arrangement may be used to detect radiation, i.e., both thermal and infrared. To this end, examples of such sensors include Geiger counter and specific isotope detectors used in laboratories or nuclear plant settings.

[0331] In other embodiments, sensors in the present arrangements include those that detect certain harmful gases, e.g., carbon monoxide, methane, and natural gas, and naturally occurring but not always easily detected harmful mold. In this context, the present teachings recognize that sensors that detect smoke may be useful when implementing methods for informing regarding certain conditions. Smoke detecting sensors are useful for emergency scenarios like firefighting and industrial safety, or ventilation planning. Similarly, light detection sensors used in the present arrangement may be used in the form of a light meter to collect accurate color intensity and temperature values from the environment. This information may be useful to the film, lighting, and interior design industries to map out the reflectivity and general photographic atmosphere of the scanned scape.

[0332] In those instances, where vibratory information is deemed useful or necessary, sensors in the present arrangement are those that detect vibrations and seismic activity. For home automation applications, the sensors of the present arrangement detect overlay locations of home appliances, HVAC controls, light switches, and power outlets. Further, based on image recognition of objects, manual entry, or triangulation of beacons, the present teachings are applied to accomplish Industrial Automation.

[0333] Regardless of whether the need is for automation, the present arrangements preferably include components that facilitate spatial data collection. To this end, the collection process preferably requires one or more users to take sensor signal measurements from various locations inside a space. More preferably, these sensor signal measurements are evenly distributed in thetarget space. By way of example, when the present teachings are implemented as a software application on an eyewear (e.g., eyewear or smartphone), sensor signal measurements may be automatically made while one or more users walk around the space. These users may either mount a sensor to the headset or AR- or XR-capable display, hold out a signal collection device within view of the headset or AR-or XR-capable display, or both.

[0334] Upon activating the signal collection mode, one or more of such users then follow onscreen instructions prompting them to walk around the area and scan it into the system. This may also be referred to as an active mode of operation. The information resulting from the active mode of operation may be stored and / or processed in the cloud or on a local cluster of graphic processing units (“GPUs”).

[0335] Before, during, or after the collection process, a calibration procedure may be invoked that requires collecting the location of certain types of sensors. By way of example, for calibration of a radio frequency (RF) antenna, a user may be instructed to precisely locate, within a real space, a known signal emitter, such as a Wi-Fi™ router, through a headset (e.g., AR or XR headset). The user may then be instructed to clockwise walk around the known signal emitter, keeping it at about an arm’s length, and then be instructed to counterclockwise walk around the same emitter.

[0336] During this procedure, the visualizing system or spatializing subsystem collects hundreds or thousands of sensor data samples at certain locations in space, while visually tracking the emitter. The system then performs a mathematical analysis on the measurements and the precise location of those measurements to model and build a 3D representation of the sensor’s signal measurements. By identifying the location of the emitter, the visualizing system or spatializing subsystem may lock onto the emitter’s exact location and then measure variations in the sensor’s signal measurement readings during the user’s circular motion. Assuming a constant emitter output and that the emitter’s signal output should remain relatively constant along a circular path around the emitter, any variations in signal collection may be attributed, with a higher degree of certainty, to the sensor’s performance and dynamics.

[0337] Cellular antennas, for example, do not emit radiation in a perfectly spherical shape, but rather in multiple overlapping elliptical shapes that may very well not be symmetrical. In this case, the present teachings recognize that carrying a protective case around the signal collection equipment, or holding it in a certain way, may also alter the antenna’s performance.

[0338] The present arrangements recognize that more than one antenna may be used during collection of sensor signal measurement. The collection process of the present teachings, in certain instances, supports multiple simultaneous users, given that they have a networked connection enabling the many users’ applications (software embodying the present teachings and installed on a client device) to communicate. When operating in multi-user collection mode, the real space is subdivided into partitions and each user is responsible for scanning their respective area, which falls within a particular partition. In the case that the two areas overlap, then the measurements from those overlapping areas may be used to improve calibration and accuracy because of the proximity of the sensor readings.

[0339] There are two modes of collection of sensor signal measurements, i.e., active mode of collection and passive mode of collection. In the active mode of collection, the user is shown a scanned outline of the space and visually directed (for example, by presenting arrows in the user’s AR or XR headset) to walk to various points for taking sensor signal measurements. Feedback representing a measurement of the sensor signal is shown on the headset as collection from different areas within a space are completed. In the passive mode of collection, on the other hand, an application (software embodying the present teachings and installed on a client device) is capable of running as a background process, “listening” for sensor signals while the user is performing another task or even playing a game. Most such applications installed on spatial computing devices, such as AR or XR headsets, continuously utilize these sensors signal scanning features. In the passive mode of collection, while other tasks are being carried out in the foreground, the sensor signal scanning for collection purposes is taking place in the background for later processing. Regardless of whether the active mode of collection or the passive mode of collection is being implemented, the collection element or process collects not only signal measurement data, but also the coordinates of the location where that signal measurement was obtained. This spatial data may be stored, in an encrypted format, if necessary, for security purposes, in a memory (for example, located on a local device with a database or log) from where it is later retrieved for processing. The spatial data may alternatively be stored on a remote server or using cloud-based synchronization. As yet another example, the present teachings may rely upon local caching of cloud data. The present teachings recognize that it is possible to retrieve existing real space measurements, which originated from other users or public or private building plans, from cloud storage.

[0340] The signal measurement data and its corresponding location coordinates may be used to perform edge and cloud GPU signal analysis, interpolation between one or more sensor signal measurements, and extrapolation of sensor signal measurement data, using algorithms and calculations per signal type, to predict sensor signal values outside a space. In certain instances, the present arrangement includes employing dedicated hardware on-device for single pass filtering and live feedback, sync to the cloud for supercomputer-class multi-pass filtering and far more advanced algorithmic processing. As mentioned above, cloud storage may also aggregate multiple users’ sensor signal measurements to fill in the gaps. For processing sensor signal data, the present teachings may rely upon CPU-based raytracing or any traditional approach of raytracing. In one embodiment, the present teachings use multiple cores by distributing equal parts of a scene or space that is being perceived by the user’s AR or VR headset. In an alternate embodiment, the present teachings use the relatively new techniques of GPU-based raytracing that perform raytracing like CPUs in the traditional approach. By way of example, when live GPU raytracing is implemented, the present teachings recognize that special GPU (repurposing game technology) may be implemented. To this end, Nvidia’s RTX hardware, which includes built-in special raytracing instructions, works well. An application (software embodying the present teachings and installed on a client device) may translate the sensor signal measurements into a format the GPU is able to recognize and understand, so that the GPU is effectively fooled into thinking it is rendering a video game scene.

[0341] The present teachings may also rely upon mobile raytracing that will enable highly accurate results even when running on battery power in the field. During such a process, the present teachings may implement such elements as storing processed data, rendering of VR, AR, or XR signals, and / or providing feedback during the sensor signal measurement collection process.

[0342] In those instances, where the present teachings rely upon one or more users to walk around the real space, the general shape of the sensor signal measured may be recorded and mapped into a spatial model by an underlying AR / VR / XR operating system of the present arrangements. This three-dimensional model or spatialized model may be made available to the system for building an approximate virtual representation of the real space’s layout and contents. This is electronically represented to the user as a “mesh,” which is overlaid on top of the detected shapes. The present system may use a similar concept for showing mapped and unmapped areasduring collection of sensor signal measurements. The present teachings recognize that when one or more people walk more of the different locations inside the space and collect attribute values more values for the different locations resulting the three-dimensional spatialized attribute value data of higher resolution, and this data is continuously updating as more people are walking to cover more area - in other words, the participants contribute to the higher resolution of the three- dimensional spatialized attribute value data).

[0343] In one aspect of the present teachings, unscanned, unmeshed areas are represented by dim, semi-transparent grids of dots / vertices that may protrude into as-yet-unmeshed elements of the real space. In another aspect of the present teachings, unscanned, meshed areas are represented by dim, semi-transparent grids of dashes or connected lines that roughly approximate the shape of the real space and its contents. In yet another aspect of the present teachings, scanned, meshed areas are shown with bold, colored vertical or horizontal lines with a brightness corresponding to the confidence of the signal measurements. In certain of these visual configurations, the present arrangements may display areas that have not been scanned, or display areas that have been scanned but are outdated, or have low confidence, or display areas that have not been scanned and may be interpolated or extrapolated, and as a result, do not need to be scanned.

[0344] In certain embodiments of the present teachings, a raw signal may be represented in the background or taking the form of “fog” and a processed signal may be represented in the background of or taking the form of “rain.” When rendering the enhanced three-dimensional spatialized environment, the present teachings may show metadata like protocol versions, device addresses, and frequencies. Other examples presented during rendering include displaying techniques for extreme-scale including instancing, point clouds, and frustum culling. In the context of a rendered enhanced three-dimensional spatialized environment, the present arrangements may allow for at least one of: interacting with a signal representation by a user; toggling between signal layers; using a graphical user interface to enable / disable multiple signal overlays; proximity-based control; activating whichever signal layers are the strongest or have the most accurate data; hand interaction for precise measurements in space; holding out hand to get extrapolated measurement at that exact point; touching the interactive signal in the form of “raindrops” to get detailed information; controlling time for historical analysis; performing transceiver activity overlay; displaying emitters; animating 3D icons are placed at the location ofdetected transmitters; displaying sensors and signal collectors, displaying signals passing between devices; animating lines of various styles including dotted and dashed are drawn between transmitters and receivers; and / or displaying the direction and general path of that signal through the air. In the case that the present system detects physical barriers and signal reflections, the lines shown will follow the model to create the most accurate display. By way of example, line segments use a combination of color, size, and frequency of animation to indicate the magnitude of signal activity such as bandwidth or latency of network communication.

[0345] In those embodiments when the present teachings rely upon a multi-user operation, fusion of sensor signal measurements obtained from multiple signal collectors on multiple users is performed. When merging or fusing sensor signal measurements obtained from multiple signal collectors, the present teachings preferably use sensor proximity as a factor in merging of fusing the sensor measurement readings into the collection process. To this end, the above-mentioned location calibration process facilitates accurately fusing sensor measurement data. By way of example, fusing or merging, according to the present teachings, relies upon a process that is similar to that performed by differential GPS systems, which uses known points for performing certain corrections. The present teaching, when implementing fusing or merging, may also use AI / ML for transceiver placement and real space-scale optimization. In other embodiments, the present teachings may carry out macro-scale aggregation for carrier-level optimization to accomplish fusing or merging as described herein.

[0346] An exemplar method for rendering gas leakage is explained below. During operation, a user may employ an AR / VR device, such as an AR / VR headset, for gas detection. The AR / VR device may be mobile or stationary relative to the user. The AR / VR device may be portable or at least in-part movably limited. The “user” is used in the context of its broadest meaning. The user may refer to a human, a machine, a tool or accessory, an artificial intelligence unit, or any suitable combinations thereof. In some examples, the user may operate, or cease to operate, either independently or in response to an indication from a remote device to perform a task. The indication may correspond to a transient signal or a non-transient signal. Examples of the indication may include, but are not limited to, audio, visual, haptic, text-based, symbolic, or any suitable combinations thereof.

[0347] The user may employ the AR / VR device in an indoor environment; however, the concepts described in this patent application may be also applied in the context of an outdoorenvironment. In one example, the user may employ the AR / VR device in an industrial environment, such as a factory warehouse. The description below describes the novel concepts for detection, analysis, and visualization of one or more gases in an ambient environment. In one embodiment, the AR / VR device may include an imaging sensor mounted thereon. The imaging sensor may include or represent a camera; however, other suitable examples may also be contemplated. For example, the imaging sensor may include light detection and ranging (LIDAR) sensors, infrared sensors, ultrasound sensors, time-of-flight sensors, ultraviolet (UV) sensors, and plenoptic sensors, or any suitable combinations thereof. The camera may operate in communication with any suitable type of processors known in the art. For example, the camera, in communication with a processor, may be configured to capture an image (or video stream) and related image data in the ambient environment. The processor may be mounted on the AR / VR device or located remote therefrom.

[0348] The camera may generate an image or a video stream of an environment. The video stream (or image) may be a single Red, Green, Blue (RGB) video stream with a predefined resolution (e.g., 1920x 1080 pixels, 640x480 pixels, 1024x768 pixels, etc.) at a preset frame rate such as 10 frames per second (fps), 20 fps, 30 fps, 40 fps, 50 fps, 100 fps, and 1000 fps. In some examples, the processor may trigger the camera to dynamically change (i.e., increase, decrease, or alternately switch between set values of) the preset frame rate and / or the preset resolution of the camera and / or the video stream (or image) based on one or more aspects including, but not limited to, remote viewing, ambient lighting conditions, motion detection, power consumption by the underlying device such as the AR / VR device connected to / with the camera, and a change in network bandwidth / congestion relative to a preset network threshold value. In one example, the network threshold value may relate to, without limitation, buffer queue length threshold, buffer occupancy threshold, maximum allowable bandwidth, packet loss rate, latency threshold, or any suitable combinations thereof. The video stream may include a sequence of images captured over time by the camera. Each image may represent a visual scene corresponding to a surrounding environment of the AR / VR device. In some examples, the video stream may include a single image. The generated video stream, or an image, may be accessible by the processor. In some examples, the video stream may be stored, e.g., in the form of a data file, in a data storage device for future access and / or retrieval by the processor.

[0349] In one embodiment, the processor may be configured to (i) measure the image data values, e.g., RGB values (or RGB-Depth values), captured by the imaging sensor such as the camera, (ii) determine spatial position of the camera and that of a remote sensor relative thereto, (iii) measure data values such as intensity values received or measured by the remote sensor, (iv) spatialize the image data values and the intensity values in a virtual space, and / or (v) render the virtual space on a display screen. In some embodiments, the processor may obtain or access a pre-stored dataset including one or more intangible attributes such as radio signals, image data values, and / or various measurements related thereto. The processor, in some examples, may be configured to identify and / or process predetermined types of intangible attributes, when received via the remote device, e.g., a non-imaging sensor, based on the intangible attribute types recorded in the prestored dataset. Alternatively, the processor may obtain or create the intangible attribute dataset depending on a type of the non-imaging sensor.

[0350] In one embodiment, the non-imaging sensor may represent a single sensor or a group of sensors. The non-imaging sensor may be configured to transmit, receive, and / or detect radio signals. In one example, the non-imaging sensor includes a software-defined radio (SDR) sensor. The SDR sensor may be implemented as a sensor device including the processor. For example, the sensor device may include radiofrequency (RF) sensors combined with the processor in a single package to transmit, receive, detect, and measure electromagnetic signals in a predefined frequency range. The sensor device (or the SDR sensor) may be mounted on the AR / VR device or located remote therefrom. The sensor device (or the SDR sensor) may be located at an offset distance relative to a spatial position of the camera in the real space. The sensor device (or the SDR sensor) may be mobile or stationary relative to the AR / VR device and / or the camera. In one embodiment, the SDR sensor may be configured as a portable handheld device. For example, the SDR sensor may be implemented as an access card or a beacon tag, which may be carried and navigated by the user in the warehouse.

[0351] The SDR sensor may be configured to mimic light sensors and / or transceivers to facilitate spectroscopy-based gas detection. In one embodiment, the sensor device (or the SDR sensor) may be configurable by the processor to operate in one or more frequency bands in the optical spectrum. Examples of the frequency bands may include, but are not limited to, 300 GHz to 800 THz, 300 GHz to 400 THz, and 400 THz to 800 THz. Since gases absorb and / or emit electromagnetic radiation at specific frequencies, the SDR sensor may be tuned by the processorto transmit, receive and / or monitor one or more frequencies in any of the tuned frequency bands to detect the presence and / or concentration of target gases.

[0352] In one embodiment, the processor may be configured to localize the camera and the SDR sensor (or the underlying sensor device) in the real space to spatialize the first type of dataset in a virtual space. The first type of dataset (or image dataset) may include the one or more image data values captured by the camera and 3D spatial coordinates in the real space at which those one or more image data values are obtained. For this, in one embodiment, the processor may implement an exemplary method Ml to spatialize the camera and the image dataset for rendering. For example, the method Ml may include a step I for obtaining boundary data of a real space. The boundary data depends on the field of view (FOV) and focal length (i.e., pixels per unit distance) of the camera. In certain embodiments, this step I of the present teachings includes obtaining three-dimensional coordinates that define, z.e., electronically represent, a boundary of a scene. A scene is commonly present in the real space in front of a user of the AR / VR device such as an eyewear.

[0353] Next, the method Ml includes a step II that requires subdividing the boundary data of the real space into a plurality of subdivisions. In certain embodiments, this step II of the present teachings involves subdividing the electronically represented boundary obtained in the previous step I, such that each of the resulting subdivisions is electronically represented by three- dimensional subdivision boundary coordinates. In other embodiments of the present teachings, subdividing described in this step II includes spatially partitioning a real space into a plurality of subdivisions. In these embodiments, a plurality of corresponding three-dimensional location coordinates collectively forms an electronic representation of the real space and this space undergoes spatial partitioning. The word “corresponding,” used in connection with term “location coordinate,” as it appears in the method Ml, conveys that the “location coordinate” is one where at least one pixelated or at least one voxelated value was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the pixelated or voxelated values obtained there.

[0354] In one embodiment, the processor may be configured to define an origin in the real space to determine real-world coordinates of each point in the environment. The origin may serve as a reference point in a 3D coordinate system. From the origin, three perpendicular axes, namely, x- axis, y-axis, and z-axis, may extend to describe a spatial position of any point in a 3D space. Theorigin may be represented as (0, 0, 0), where the x-axis, y-axis, and z-axis intersect at a single point in space. The origin may be set at any suitable point in an indoor location by the processor. In one example, the origin may correspond to a corner of a room, such as the warehouse, where the x-axis may extend along a length of the room (e.g., left to right), the y-axis may represent a height (e.g., floor to ceiling) of the room, and the z-axis may extend from the back to a front of the room. In some examples, the origin may correspond to or defined relative to global positioning system (GPS) coordinates for an outdoor location.

[0355] Further, the processor may be configured to determine a position of the camera relative to the origin. For example, the processor may determine the camera to be located at a spatial coordinate (3, 2, 5), such that the camera may be located 3 meters from the origin along the x- axis, 2 meters from the origin along the y-axis, and 5 meters from the origin along the z-axis in a world coordinate system (i.e., in the real space). Other examples may include the processor setting a starting position of the camera at the origin to define that the camera is located at the origin. In one embodiment, the camera may capture a scene in its field of view as an image and use the captured image to create a virtual space. The camera’s position (or starting position in some examples) in the virtual space may correspond to an origin of the virtual world coordinate system, with the camera looking along the negative z-axis (e.g., OpenGL convention) or positive Z-axis (e.g., DirectX convention), the x-axis moving left to right, and the y-axis moving up and down on the screen. In the virtual space, the camera may be deemed located at 3D coordinates (0, 0, 0) representing the origin in the virtual space. In one example, the objects further from the camera may have increasing z-axis values.

[0356] The method Ml involves a step III, which includes obtaining three-dimensional pixelated data values or three-dimensional voxelated data values for corresponding location coordinates. In this step III, each of the corresponding location coordinates is “associated” with at least one of associated three-dimensional pixelated data values or at least one associated three-dimensional voxelated data values. The word “associated,” used in connection with the terms “three- dimensional pixelated data values” and “three-dimensional voxelated data values,” conveys that the pixelated or the voxelated values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0357] In one example of this step III, obtaining the three-dimensional pixelated data values or the three-dimensional voxelated data values includes measuring light intensity or measuringcolor intensity present at a unit pixel area or a unit voxel (volumetric) space, respectively, of an optical sensor, such as the camera. According to one embodiment of the present teachings, step III is carried out, e.g., using an optical sensor or a database to obtain the required data values may be similarly implemented in step III. For example, the camera may be configured to measure a predefined first attribute in the real space. For instance, the user may navigate the AR / VR device, e.g., in the warehouse, for investigating or inspecting a gas leak. The navigating AR / VR device may have the camera capture the predefined first attribute, such as image data, at one or more three-dimensional (3D) spatial coordinates in the real space. The 3D spatial coordinates may define a region or location in the real space. The image data may include at least one of pixelated data values and voxelated data values. The pixelated data values may correspond to values of pixels in a two-dimensional (2D) image. Each pixel value may represent a color or an intensity at a specific point in a 2D image. Similarly, the voxelated data values may correspond to values of voxels (i.e., volumetric pixels) in a 3D image. Each voxel may represent volumetric information such as density or material properties at a specific point within a 3D image. In one example, at a given time instant Tl, the imaging sensor such as the camera may capture image data value Al at spatial coordinate (XI, Yl, Zl) in the real space. Similarly, at given time instant T2, the camera may capture image data value A2 at the same spatial coordinate (XI, Yl, Zl) in the real space, and so on. Even at the same spatial coordinate, namely, (XI, Yl, Zl), the image data values captured by the camera may differ at different time instants due to a change in ambient lighting conditions and / or objects being moved within the field of view of the camera.

[0358] In one embodiment, the camera may capture RGB color values for each pixel, where each of the Red (R) color values, Green (G) color values, and Blue (B) color values may range from 0 (e.g., corresponding to absence of color, or black) to 255 (e.g., corresponding to maximum intensity of color). In one example, each of the image data values, e.g., Al to A20, may be represented in the (R, G, B) format. For instance, the image data values Al and A2 may be (10, 50, 200) and (55, 100, 250) respectively, where the R values may be 10 and 55, the G values may be 50 and 100, and the B values may be 200 and 250. Alternatively, the image data values Al to A20 may be represented in (Gr) format, where Gr is a single scalar intensity value ranging from 0 (e.g., corresponding to black) to 255 (e.g., corresponding to white) for a grayscale image (or video stream). Other image data values from A3 to A20 may also be represented in the (R, G, B)format or (Gr) format. In some examples, the image data values may correspond to the voxelated data values, which may include or be associated with the pixelated data values corresponding to a portion of the image. The voxelated data values may be represented as the single scalar intensity value, the (R, G, B) format, the (R, G, B, A) format, where A refers to transparency or opacity having a value ranging from 0 (e.g., corresponding to fully transparent) to 255 (e.g., corresponding to fully opaque), or the (R, G, B, D) format, where D refers to depth or distance from the camera to a point in space.

[0359] Once the subdivisions are obtained, the method Ml may proceed to step IV, which involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and step IV generates one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisions (i.e., resulting from step II) that include at least one three-dimensional location coordinate, i.e., at least one location from where a measurement of the pixelated or the voxelated values was obtained.

[0360] Then, method Ml advances to a step V that includes assigning at least one of the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values to one or more of the selected subdivisions to define one or more assigned subdivisions. In this step V, each of the three-dimensional pixelated data values or the associated three-dimensional voxelated data values, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated three- dimensional pixelated data values or at least one of the associated three-dimensional voxelated data values.

[0361] In one preferred aspect, step V includes assigning at least one of the associated three- dimensional pixelated data values or at least one of the associated three-dimensional voxelated data values to an entire portion of the selected subdivisions. In another preferred aspect, where two or more pixelated or voxelated values are available, step V includes assigning a summation of weighted values of two or more of the associated three-dimensional pixelated data values or a summation of weighted values of two or more of the associated three-dimensional voxelated data values to the selected subdivisions.

[0362] After conclusion of step V, the method Ml preferably proceeds to a step VI, which involves integrating the assigned subdivisions to form a spatialized image dataset. In certain preferred embodiments, step VI of the present teachings includes integrating a plurality of the three-dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized image dataset. In other preferred embodiments, step VI of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated three-dimensional pixelated data values or the associated three-dimensional voxelated data values are distributed, based upon the spatialized model, to create the spatialized image dataset.

[0363] Further, the processor may be configured to map the camera’s position from the real space to the virtual space by applying various transformations including translation, rotation, scaling, and projection. The translation moves the camera from the real -world coordinates to its designated position in the virtual space. The rotation orients the camera to match real-world orientation. The scaling adjusts sizes of various objects to fit within the virtual space. The projection maps 3D real world coordinates to 2D image plane coordinates. The processor may use a predefined transformation matrix to translate and rotate the camera from the real world to the virtual space. The processor may also employ a predetermined projection matrix configured to combine intrinsic parameters and extrinsic parameters to map real-world coordinates (X, Y, Z) to the image plane coordinates (X0, Y0) in the virtual space. The intrinsic parameters may include camera’s focal length, optical center of an image, and an aspect ratio of the image / camera. The extrinsic parameters may include the rotation of a target coordinate system (e.g., world coordinate system, sensor coordinate system, etc.) to conform to camera’s orientation and the translation of coordinates in the target coordinate system to camera’s position in the virtual or real space.

[0364] The processor may be further configured to spatialize the offset distance relative to the camera to localize the SDR sensor in the real space, where the SDR sensor may be located at the offset distance from the camera. In one embodiment, the processor may determine a spatial position of the SDR sensor (or the underlying device) in the real space relative to the camera and map the SDR sensor to the virtual space. For example, the camera may detect or capture the SDR sensor when present in the camera’s field of view. The processor may determine a relative position of the SDR sensor and its offset distance from the camera based on a reference markerin the camera space. The reference marker may correspond to a common spatial feature (e.g., vertex, longitude or latitude) or a common visual feature (e.g., augmented reality (AR) marker, checkerboard, objects, etc.) scanned or detected by both the camera and the SDR sensor. Alternatively, when the SDR sensor (or the underlying device) may be located remote from the camera, the processor may determine and generate the offset distance based on the strength of signal generated by the SDR sensor and received via a reference sensor. Other examples may include the offset distance between the camera and the SDR sensor being known or predetermined.

[0365] In one embodiment, the processor may use at least one of (i) a signal strength and / or a frequency of the radio signals and / or (ii) a predetermined frequency band to selectively process one or more of the radio signals. In one example, the processor may be preconfigured or dynamically configured to tune the SDR sensor to transmit radio signals in different frequency bands in the optical spectrum. For instance, the processor may tune the SDR sensor to transmit radio signals of various frequencies in the optical spectrum. In some instances, the SDR sensor, upon being tuned by processor, may sequentially switch between different frequency bands in the optical spectrum to transmit and / or receive radio signals at different frequencies. The processor may tune the SDR sensor to transmit and / or receive radio signals at set frequencies in each of the predefined frequency bands for a set period.

[0366] The radio signals transmitted, by the SDR sensor, at a frequency belonging to a first set a of one or more frequencies (or frequency bands) may excite the ambient gases. Upon being excited, one or more ambient gases may emit electromagnetic radiations that may be received as radio signals (or energy signals) by the SDR sensor. The radio signals (or energy signals) may be received by the SDR sensor at a different frequency belonging to a second set of one or more frequencies. In some examples, the radio signals (or energy signals) may be received upon reflection from the ambient environment by the SDR sensor. The received radio signals (or energy signals) may be measured by the processor to determine the corresponding signal strength or intensity values.

[0367] In one embodiment, the processor may transform each of the one or more radio or energy signals received by the SDR sensor into frequency domain to produce a transformed signal. Each respective transformed signal of the one or more received signals may include one or more peaks corresponding to frequencies in a frequency band within the optical spectrum. The receivedsignals may be transformed by the processor using any suitable algorithms known in the art including, but not limited to, Fast Fourier Transform (FFT), Discrete Fourier Transform (DFT), and Discrete Cosine Transform (DCT). The transformed signal (in frequency domain) may include or be divided into one or more frequency bands within the optical spectrum.

[0368] The transformed signal may include one or more frequencies in one or more frequency bands. Examples of the frequency bands may include, but are not limited to, 1-30 THz, 30-60 THz, 60-90 THz, and 90-120 THz. The number and widths of these frequency bands of interest may be configured using the processor based on (i) a number and / or types of target gases and (ii) the intended processing complexity and / or a battery usage of the processor and / or the underlying device (e.g., remote device, sensor device, AR / VR device, etc.). Each of the frequency bands may include one or more frequency peaks based on the received signals. Each of these frequency peaks may correspond to a frequency at which the received radio / energy signal may have an intensity value (or signal strength) equal to or greater than a predefined signal threshold value. The processor may be configured to ignore and / or set as noise those frequencies at which the radio / energy signal may have an intensity value (or signal strength) below or less than such predefined signal threshold value. In the frequency domain, a group of one or more peaks above the predefined signal threshold value may form an energy cluster.

[0369] The processor may be configured to detect the presence of a target gas based on one or more energy clusters within a frequency band. For example, ammonia gas exhibits absorption bands near a wavelength of 10 micrometer (herenafter “pm”) (or 10000 nanometer), associated with N-H bending vibrations. The wavelength of 10 pm (i.e., 10000 nanometer) corresponds to a target frequency of approximately 30 Terahertz (hereinafter “THz”). Similarly, the methane gas has an absorption wavelength of 3300 nanometer (nm), which corresponds to a target frequency of approximately 90.9 THz. Further, the carbon dioxide in the gaseous form has an absorption band around 4260 nm (corresponding to a target frequency of approximately 70.4 THz) and 2700 nm (corresponding to a target frequency of 111.1 THz), the carbon monoxide gas has an absorption near 4670 nm (corresponding to a target frequency of approximately 64.2 THz), the sulfur dioxide gas has an absorption near 7300 nm (corresponding to a target frequency of approximately 41.1 THz), the hydrogen sulfide gas has an absorption near 3900 nm (corresponding to a target frequency of approximately 76.9 THz). Also, the ozone gas has a significant absorption band around 9600 nm (corresponding to a target frequency ofapproximately 31.25 THz), and a strong absorption around 250 nm (corresponding to a target frequency of approximately 1200 THz) and 330nm (corresponding to a target frequency of approximately 909 THz).

[0370] In one embodiment, when a set minimum number (e.g., at least two) of frequency peaks corresponding to a target frequency are above the predefined signal threshold value, the processor may detect the presence of a gas associated with the target frequency. For example, the processor may detect the presence of ammonia gas when at least two frequency peaks at the target frequency of 30 THz may be equal to or greater than a predefined signal threshold value. The minimum number of frequency peaks at the target frequency may be obtained at different time instants within a set period and / or at different spatial coordinates in the real space. For example, when a first frequency peak equal to or greater than a predefined signal threshold value may be obtained at the spatial coordinates (XI, Yl, Zl) in the real space and a second frequency peak equal to or greater than the predefined signal threshold value may be obtained at the spatial coordinates (X2, Y2, Z2) in the real space, both at the same target frequency of 30 THz, the processor may detect the presence of ammonia gas in the vicinity. In another example, when a first frequency peak equal to or greater than a predefined signal threshold value at a target frequency of 30 THz may be obtained at a first time instant T1 and a second frequency peak equal to or greater than the predefined signal threshold value at the same target frequency of 30 THz may be obtained at a second time instant T2, both at the same spatial coordinates (XI, Yl, Zl) in the real space and within a set period (e.g., 1 second, 2 seconds, 5 seconds, etc.), the processor may detect the presence of ammonia gas in the vicinity. Similarly, in a further example, the processor may detect the presence of Ozone gas in the vicinity when two frequency peaks equal to or greater than a predefined signal threshold value may be detected at the target frequencies of 1200 THz and 909 THz within a set period and / or different spatial coordinates in the real space.

[0371] The processor may be configured to record a frequency domain dataset including one or more frequency peak values and the corresponding 3D spatial coordinates in the real space at which these frequency peak values may be observed or detected by the SDR sensor. The frequency peak values in the frequency domain dataset may be equal to or greater than one or more predefined signal threshold values. Each frequency peak value in the frequency domain dataset may correspond to a target frequency associated with a target gas from a group of gasesincluding, but not limited to, ammonia, carbon dioxide, carbon monoxide, methane, surfer dioxide, hydrogen sulfide, and ozone.

[0372] In one embodiment, the processor may be configured to spatialize the SDR sensor and related data (i.e., the frequency domain dataset) for rendering. For this, one exemplar method M2 may begin with a step SI obtaining boundary data of a real space. In certain embodiments, a step SI of the present teachings includes obtaining three-dimensional coordinates that define, i.e., electronically represent, a boundary of a scene. Step SI is substantially similar to the step I mentioned above for the camera and the different embodiments described with respect to step I may similarly be implemented to carry out step SI.

[0373] Next, the method M2 includes a step SII that requires subdividing the boundary data of the real space into a plurality of subdivisions. Step SII is substantially similar to step II and the different embodiments described with respect to step II may similarly be implemented to carry out step SII. By way of example, step SII of the present teachings may involve subdividing the electronically represented boundary obtained in step SII, such that each of the resulting subdivisions is electronically represented by three-dimensional subdivision boundary coordinates. As another example, subdividing of step SII may include spatially partitioning the real space into a plurality of subdivisions. In this example, a plurality of corresponding three- dimensional location coordinates collectively forms an electronic representation of the real space that undergoes spatial partitioning. The word “corresponding,” used in connection with term “location coordinate,” as it appears in this method M2, conveys that the “location coordinate” is one where at least one attribute value of a particular type was obtained. Hence, the “location coordinate,” “corresponds” to, or has a connection with, the attribute value obtained there.

[0374] This method M2 also includes a step Sill, which requires obtaining one or more different types of attribute values for corresponding location coordinates. In this step, each of the corresponding location coordinates is “associated” with at least one type of attribute values. The word “associated,” used in connection with the terms “attribute values,” conveys that one or more attribute values, obtained at a particular location coordinate, are “associated” with that location coordinate.

[0375] In one example of step Sill, a magnitude of an intangible property of a real space is obtained. In one implementation of this example, step Sill includes detecting, inside the real space, a value of a parameter, e.g., magnitude, of at least one type of attribute that is chosen froma group comprising throughput of a connectivity signal, latency of a connectivity signal, interference of a connectivity signal, volatility of a connectivity signal, stability of a connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature. In one embodiment, the processor may be configured to determine a concentration of a target using Equation 15.where:I = Received intensity value (e.g., received signal strength of radio signal) after passing through a target gasIo= Initial intensity value (e.g., transmitted signal strength of radio signal) before entering the gasL = path length (e.g., distance between the SDR sensor coordinate to target gas source coordinate) u = absorption cross-section, a constant specific to the gas and the underlying target frequency

[0376] In Equation 15, the absorption cross-section a relates to the absorption coefficient (of the target gas) which is directly proportional to the gas concentration.

[0377] According to one embodiment of the present teachings, step Sill is carried out in a manner that is substantially similar to step II discussed above. Further, the different embodiments, e. ., using a non-optical sensor such as the SDR sensor or a database to obtain the required data values, described in connection with step II above, may be similarly implemented in step Sill.

[0378] Once the subdivisions are obtained, this method M2 may proceed to step SIV, which involves identifying one or more of the subdivisions that contain one or more of the corresponding location coordinates. Such subdivisions are, preferably, referred to as “selected subdivisions,” and step SIV produces one or more of such selected subdivisions. In other words, selected subdivisions simply refer to those subdivisions (z.e., resulting from step SII) that include at least one three-dimensional location coordinate, z.e., at least one location from where ameasurement of at least one attribute value of a particular type (e.g., frequency peak value) was obtained.

[0379] Then, this method M2 advances to a step SV that includes assigning at least one of the associated attribute values of at least one type to one or more of the selected subdivisions to define one or more assigned subdivisions. In this step, each of the associated attribute values of a particular type, which are associated with at least one of the corresponding location coordinates, are assigned to the selected subdivisions. Further, after the assignment, these selected subdivisions are referred to as the “assigned subdivisions.” Stated another way, each assigned subdivision is assigned at least one of the associated attribute values of a particular type.

[0380] In one preferred aspect, step SV includes assigning at least one of the associated attribute values of a particular type to an entire portion of the selected subdivisions. In another preferred aspect, where two or more attribute values of a particular type are available, step SV includes assigning a summation of weighted values of two or more of the associated attribute values of a particular type to the selected subdivisions.

[0381] After conclusion of step SV, this method M2 preferably proceeds to a step SVI, which involves integrating the assigned subdivisions to form a spatialized attribute value dataset (e.g., frequency domain dataset). In certain preferred embodiments, step SVI of the present teachings includes integrating a plurality of the three-dimensional subdivision boundary coordinates, which define one or more subdivisions, to form the spatialized attribute value dataset (e.g., frequency domain dataset). In other preferred embodiments, step SVI of the present teachings includes integrating the plurality of subdivisions to create a spatialized model for the real space. In these embodiments, the associated attribute values of at least one type are distributed, based upon the spatialized model, to create the spatialized attribute value dataset.

[0382] Further, the processor may be configured to spatialize the frequency domain dataset to create a second type of spatialized dataset that spatially distributes one or more of the frequency peak values for rendering the frequency domain dataset. In one embodiment, the processor may be configured to measure an intensity value of a radio signal received by the SDR sensor at a first location LI having coordinates (XI, Yl, Zl) in the real space. The user may navigate the SDR sensor (or the underlying device) to a second location L2 having coordinates (X2, Y2, Z2) in the real space, and then to a third location L3 having coordinates (X3, Y3, Z3) in the realspace. The processor may also measure the respective intensity values of radio signals received by the SDR sensor at each of the locations L2 and L3.

[0383] In one embodiment, the processor may transform the received signals into frequency domain, as discussed above, and measure frequency peaks values corresponding to a target frequency associated with a target gas. If the frequency peak values may be equal to or greater than a predefined signal threshold value at a minimum of three spatially different locations, such as locations LI, L2, L3 in the real space, the processor may detect and verify the presence of the target gas source by employing a trilateration process to determine the coordinates of a location T, where a source of the target gas may be located. For example, the processor may measure a first frequency peak Pl value being equal to or greater than a predefined signal threshold value at a target frequency of a signal received by the SDR sensor at the first location LI, as discussed above. The processor may also determine a Euclidean distance (dl) between the locations LI and T based on the received signal strength (in dBm or decibel-milliwatts) of the signal received by the SDR sensor at location LI, using Equation 16. dl = (RSSIth- RSSIL1)2(16) where: dl = Euclidean distance between the locations LI and T, i.e., signal source location RSSIth= Predefined / stored signal threshold value associated with a target frequency RSSIL1= signal strength of a signal received by SDR sensor at location LI (in dBm)

[0384] As the user may navigate the SDR sensor (or the underlying device) to the location L2, the processor may measure a second frequency peak P2 value equal to or greater than a predefined signal threshold value at the target frequency of a signal received by the SDR sensor at the second location L2, as discussed above. The processor may also determine a Euclidean distance (d2) between the locations L2 and T based on the received signal strength of the signal received by the SDR sensor at location L2, using Equation 17.

[0385] d2 = / (RSSItfl- RSSIL22(17) where: d2 = Euclidean distance between the locations L2 and T, i.e., signal source location RSSIth= Predefined / stored signal threshold value associated with a target frequency RSSIL2= signal strength of signal received by SDR sensor at location L2 (in dBm)

[0386] Similarly, when the user navigates the SDR sensor (or the underlying device) to the location L3, the processor may measure a third frequency peak P3 value equal to or greater than a predefined signal threshold value at the target frequency of a signal received by the SDR sensor at the second location L3, as discussed above. The processor may also determine a Euclidean distance (d3) between the locations L3 and T based on the received signal strength of the signal received by the SDR sensor at location L3, using Equation 18. d3 = -J(RSSIth- RSSIL3)2(18) where: d3 = Euclidean distance between the locations L3 and T, i.e., signal source location RSSIth= Predefined / stored signal threshold value associated with a target frequency RSSIL3= signal strength of signal received by SDR sensor at location L3 (in dBm)

[0387] Since all frequency peaks Pl, P2, P3 may be obtained at the target frequency, the processor may employ the generalized Euclidean distance formula of Equation 19 to set up equations for the trilateration process to determine the coordinates of the location T, where the target gas source may be located.where: d = Euclidean distance between a known location E (XQ, Yo, Zo) and an unknown location of the target gas source T (X, Y, Z)

[0388] Based on Equation 19, the processor may determine Euclidean distance from the location LI to T, as shown in Equation 20.where: dl = Euclidean distance between the known location LI X1, Y±, Z±) and the unknown location T of the target gas source (X, Y, Z)

[0389] Based on Equation 19, the processor may also determine Euclidean distance from location L2 to T, as shown in Equation 21. d2 = V(X - X22+ (E - Y2)2+ (Z - Z2)2(21) where:d2 = Euclidean distance between the known location L2 (X2, Y2, Z2) and the unknown location T of the target gas source (X, Y, Z)

[0390] Based on Equation 19, the processor may determine Euclidean distance from location L3 to T, as shown in Equation 22.

[0391]

[0392] where:

[0393] d3 = Euclidean distance between the known location L3 (X3, Y3, Z3) and the unknown location T of the target gas source (X, Y, Z)

[0394] Based on Equations 15-19 and 20-22, the processor may determine the 3D spatial coordinates (X, Y, Z) of the location T, where the target gas source may be located in the real space. Thus, the processor may employ the trilateration process to spatialize the various frequency peaks of the frequency domain dataset in the real space. The spatialized frequency peaks create a second type of spatialized dataset including (i) 3D spatial coordinates of various locations, such as location T, where a source of a target gas may be located in the real space, and (ii) the corresponding frequency peak values at a target frequency associated with the target gas. Thus, the second type of spatialized dataset may spatially distribute one or more of the frequency peak values in the real space.

[0395] The processor may calibrate the real-space positions of the camera and SDR sensor (or the underlying device) relative to a common reference marker (e.g., common spatial feature or a common visual feature) as discussed above. Upon calibration, to align the SDR sensor data (i.e., second type of spatialized dataset) with the camera’s view, the processor may employ the transformation matrix, discussed above, to transform the coordinates of location T (where the target gas source may be located) to the camera space, i.e., a camera coordinate system. The transformation matrix may move the location T from the sensor coordinate system to the camera coordinate system to assist in displaying the location T in the virtual space.

[0396] The processor, via the rendering engine, may use the projection matrix, discussed above, to project the 3D coordinates of the location T on to the 2D virtual image plane using camera’s intrinsic parameters. The projected 2D points may be rasterized to form a pixel-based image. The processor may compute which pixels may correspond to the 3D spatial coordinates, then applies textures, shading, and lighting, and produces the final image.

[0397] In one embodiment, the determined target gas location T may have coordinates (XT, YT, ZT) in the camera coordinate system. The processor may be configured to project this determined target gas location T to a point pixel P having coordinates (it, v) in the virtual space, using Equation 23 and Equation 24.where: fx= focal length of the camera along the x-axis fy= focal length of the camera along the y-axis cx= x-coordinate of a center of an image created by the camera, where the image forms a part of the virtual space cy= y-coordinate of the center of the image created by the camera, where the image forms the part of the virtual space(u, v) = coordinates of the point P in the virtual space, where the point P corresponds to a point having coordinates (XT, YT, ZT) in the real space (e.g., target gas location T) in the camera coordinate system

[0398] In Equations 23 and 24, the focal length of the camera and the coordinates of the center of the image may depend on a resolution of the camera. For example, for a camera having a resolution of 1920 x 1080 pixels, the focal length may be (530, 530) (e.g., for Logitech™ webcam) or (1395, 1395) (e.g., for Intel™ RealSense™ camera), and the coordinates of the center may be (960, 540). The processor, in some examples, may be configured to use Equations 9 and 10 for mapping any other 3D spatial coordinates located in the camera coordinate system, i.e., present or transformed into the field of view of the camera, to pixel coordinates in an image of the virtual space. For example, the processor may use Equations 9 and 10 with the 3D spatial coordinates of the SDR sensor in the camera coordinate system, as discussed above, to map the SDR sensor into the virtual space.

[0399] In one embodiment, the processor may be further configured to apply textures, color shadings or gradients, and / or symbols to a gas region in the virtual space, where the gas regionmay correspond to a polygonal spatial region formed by pixel coordinates corresponding to the spatial coordinates LI, L2, L3 and T. In some examples, the processor may shade a gas region in the virtual space, where the gas region may relate to a pixel region (in the virtual space) corresponding to a spatial region formed by any two of real space spatial coordinates of the camera, such as LI and L2, LI and L3, or L2 and L3, and the location T of the target gas source. The gas region in the virtual space may assist in visualizing a spread of the target gas to inspect or investigate a potential gas leakage. In some examples, a concentration of the target gas calculated by the processor using Equation 1 may also be shown proximate the gas region in an image in the virtual space. Other examples may include the processor configured to select a target frequency from a set of one or more target frequencies based on an input from a user or remote device for visualizing the gas region corresponding to a target gas in the virtual space on the display screen where the target gas may be associated with the selected target frequency.

[0400] The exemplar code provided below may be used to carry out various embodiments of the present teachings described herein.

[0401] Generate sample microphone (i.e., non-imaging) sensor data:# Generate sample microphone dataset def create mi crophone_dataset(n_sampl es= 10) : np . random . seed(42) data = { 'model_name': [fMic{i+l }' fori inrange(n_samples)], 'type': np.random.choice(['Condenser', 'Dynamic', 'USB', 'Ribbon'], n samples), 'frequency_response_low': np. random. randint(20, 50, n_samples), 'frequency_response_high': np.random.randint(15000, 20000, n_samples), 'sensitivity _db': -1 *np.random.uniform(30, 60,n_samples).round(l), 'impedance_ohms': np. random. randint(50, 600, n_samples), 'max spl db': np. random. randint(l 20, 160, n samples), 'self_noise_db': np. random. uniform(10, 25, n_samples).round(l),# Add frequency response measurements (simulated) frequencies = [20, 50, 100, 250, 500, 1000, 2000, 4000, 8000, 16000] for freq in frequencies:# Simulate frequency response with some variation around OdB data[ffreq_{freq}hz'] = np .random. normal(0, 2, n samples).round(l)# Create DataFrame df = pd.DataFrame(data)Add some common patterns df.loc[df['type'] == 'Condenser', 'sensitivity _db'] -= 5 # Condensers tendto be more sensitive df.loc[df['type'] == 'Ribbon', 'max spl db'] -= \Q#Ribbonmics often have lower max SPL return df# Create sample dataset mi c s df = create mi crophone dataset()# Example analysis print("\nSample Microphone Dataset: ") print(mics_df[['model_name', 'type', ’sensitivity_db', 'max_spl_db']].head()) print("\nAverage specs by microphone type:") print(mics_df.groupby('type').agg({'sensitivity_db':'mean','max spl db':'mean','self_noise_db':'mean'}).round(l))

[0402] Fast Fourier Transform to analyze microphone data: import numpy as np from scipy import signal import matpl otlib.py plot as pit def simulate_mic_response(duration=1.0, sample_rate=48000):. Simulate a microphone recording of a frequency sweep t = np.linspace(0, duration, int(sample_rate * duration))# Create a frequency sweep from 20Hz to 20kHz fO, fl = 20, 20000sweep = signal. chirp(t, fD=fO, fl=fl, tl=duration, method-logarithmic')# Simulate mi crophone frequency responseAdd some resonances and roll-offs typical of microphones freqs = np.fft.fftfreq(len(t), l / sample_rate) freq_response = np.ones_like(freqs, dtype=complex)# Add low frequency roll-off ( ommon in cardioid mics) freq_response[abs(freqs)< 50] *= 0.7# Add slight presence boost around 5kHz (commoninvocalmics) boost freq = 5000 boost_width= 1000 boost_mask = (abs(freqs) > (boost_freq - boost_width / 2)) & (abs(freqs) < (boost_freq+ boost_width / 2)) freq_response[boost_mask] *= 1.4# Apply frequency response to sweep sweep fft = np.fft.fft(sweep) modified_sweep = np.fft.ifft(sweep_fft * freq_response).real return t, modified_sweep, sweep def analyze_frequency_response(signal, sample_rate=48000, window_size=2048):. Analyze frequency response using FFT# Apply Hanning window to reduce spectral leakage window = np.hanning(len(signal)) windowed_signal = signal * window# Compute FFT fft = np.fft.fft(windowed_signal) freqs = np.fft.fftfreq(len(signal), l / sample_rate)# Convert to magnitude in dB m agnitude_db = 20 *np.logl 0(np . ab s(fft) + le- 10)# Add small number to avoid log(0)# Keep only positive frequencies positive freqs mask = freqs >= 0 freqs = freqs[positive_freqs_mask] magnitude db = magnitude_db[positive_freqs_mask] returnfreqs, magnitude db# Generate and analyze sample data t, modified_sweep, original_sweep = simulate_mic_response() freqs, mag_db = analyze_frequency_response(modified_sweep) orig freqs, orig mag db = analyze_frequency_response(original_sweep)# Plot the results plt.figure(figsize=(12, 6)) plt.semilogx(freqs[l :], mag_db[l :] - orig mag dbfl :])# Remove DC (0 Hz) and normalize plt.grid(True) plt.xlabel('Frequency (Hz)') plt.ylabel('Magnitude (dB)') pit. title(' Simulated Microphone FrequencyResponse') plt.xlim(20, 20000) plt.ylim(-12, 6)# Add frequency markers markers = [20, 50, 100,200, 500, 1000, 2000, 5000, 10000,20000] plt.xticks(markers, [str(m) form in markers])

[0403] Spatializing the microphone data import numpy as np import pandas as pd from scipy import signal import matplotlib .pyplot as pit from mpl_toolkits.mplot3d import Axes3D def generate_p ol ar _p attern(pattern_ty p e- cardi oi d' ) :. Generate polar pattern data for different microphone types# Generate angles from Oto 360 degrees angles=np.linspace(0, 2*np.pi,360) if pattem_type == 'cardioid': il Cardioid pattern: r = 0.5 + 0.5*cos(theta) response = 0.5 +0.5*np.cos(angles) elif pattern_type =='figures': / / Figure-8 pattern: r = cos(theta) response = np ab s(np . cos(angles)) elif pattem type == 'omnidirectional':# Omnidirectional pattern: r = 1 response = np.ones_like(angles) elif pattem type == 'supercardioid':# Supercardioidpattern: r = 0.37 + 0.63 *cos(theta) response = 0.37 + 0.63 *np. cos(angles) return angles, response def create_spatial_frequency_response(angles, frequencies, pattem_type='cardioid'):> Create frequency response data for different angles# Initialize 2D array for frequency response at different angles spatial_response = np.zeros((len(angles), len(frequencies))) it Get basic polar pattern_, basic pattern = generate_polar_pattem(pattem_type) if Create frequency-dependent response fori, angle in enumerate(angles): it Base response from polar pattern base response = basic_pattem[i]# Add frequency-dependent effects for j, freq in enumerate(frequencies): response = base response# Add high-frequency directional narrowing ( ommon in real mics) if freq > 5000: response *=np.exp(-abs(angle-np. pi) * (freq - 5000) / 20000) it Add low-frequency omnidirectional tendency iffreq < 500: response = response * (freq / 500) + (l - freq / 500) * 0.8 spatial responsefi, j] = response return spatial response defvisualize_3d_polar_pattem(pattem_type='cardioid'):> Create 3D visualization of polar pattern with frequency response# Generate frequency and angle arrays frequencies = np.logspace(np.logl0(20), np. log 10(20000),100) angles = np.linspace(0, 2*np.pi, 180) it Create spatial frequency response data spatial_response = create_spatial_frequency_response(angles, frequencies, pattem_type)Il Create meshgrid for 3D plot angle grid, freq grid = np.meshgrid(angles, frequencies)X= spatial response.! * np.cos(angle_grid)Y = spatial response.T * np.sin(angle grid)Z = np.log 10(freq_grid) # Logfrequency scale# Create 3D plot fig = plt.figure(figsize=(12, 8)) ax = fig.add_subplot(l 11, projection='3d')# Plot surface surf = ax.plot_surface(X, Y, Z, cmap-viridis1, alpha=0.8)# Customize plot ax.set xlabel('X') ax.set_ylabel('Y) ax.set_zlabel('Frequency (Hz)1) ax.set_title(f3D Spatial Response - {pattem_type.capitalize()} Pattern')Add color bar fig.colorbar(surf, label-Relative Response (dB)')# Setfrequency ticks freq_ticks=[20, 100, 1000, 10000,20000] ax. set_zticks(np. log 10(freq ticks)) ax.set_zticklabels([str(f) for fin freq_ticks]) return fig def analyze_spatial_response(angle, frequency, pattem_type='cardioid'):. Analyze response at specific angle and frequency angles =np.linspace(0, 2*np.pi, 180) frequencies = np.logspace(np.logl0(20), np.logl 0(20000), 100)# Get full spatial response spatial_response = create_spatial_frequency_response(angles, frequencies, pattem_type)# Find nearest indices angle_idx = np.abs(angles - angle). argmin() freq idx =np.abs(frequencies - frequency) . argmin() return spatial_response[angle_idx, freq_idx]# Example usage patterns = ['cardioid', 'figure8', 'omnidirectional', 'supercardioid'] forpattern in patterns: visualize_3 d _polar_pattem(patte m) plt.show()# Example analysis at specific points test_angle = np.pi / 4 # 45 degrees test_freq = 1000 #1kHz for pattern in patterns : response = analyze_spatial_response(test_angle, test_freq, pattern) print(f'{pattem.capitalize()} response at {test_angle*180 / np.pi:.lf}° and{test_freq}Hz: {response:.3f}")

[0404] Aligning and render imaging data with microphone data. import numpy as np import matplotlib.pyplot as pit from matplotlib .patches importCircle, Wedge import cv2 from scipy.ndimage import gaussian filter class SpatialAudioVisualizer: def init (self, image_size=(720, 1280)): self.image_size = image_size self.height, self, width = image_size def create_simulated_depth_map(self) :. Create a simulated depth map (usually from stereo cameras / RGBD sensor) y, x = np . ogri d [ : self . height, : self . wi dth] center_y, center_x = self.height / 2, self.width / 2Add some random variations for texture noise = np. random. normal(0, 0.1, (self.height, self. width)) depth map += noise# Smooth the depth map depth_map = gaussian_filter(depth_map, sigma=5)# Normalize to 0-1 range depth_map = (depth_map - depth_map.min()) / (depth_map.max() - depth_map.min())return depth map def generate_mic_pattem(self, pattem_type='cardioid', frequency=1000): Generate microphone polar pattern response theta=np.linspace(0,2*np.pi, 360) if pattem_type == 'cardioid' : response = 0.5 + 0.5*np.cos(theta) elif pattern_type == 'figure8': response = np.abs(np.cos(theta)) elif pattem type == 'supercardioid' : response = 0.37 + 0.63 *np.cos(theta) else: # omnidirectional response = np.ones_like(theta)# Add frequency-dependent effects if frequency > 5000: response *=np.exp(-np.abs(theta-np.pi) * (frequency- 5000) / 20000) elif frequency < 500: response = response * (frequency / 500) + (l -frequency / 500) * 0.8 return theta, response def create_overlay_visualization(self, mic_position=(0.5, 0.5), pattern_type='cardioid', frequency=1000):> Create visualization with depth map and microphone pattern overlay# Create depth map depth map = self.create_simulated_depth_map()# Create figure fig, ax = plt.subplots(figsize=(15, 8))# Display depth map depth img = ax.imshow(depth map, cmap-viridis') plt.colorbar(depth img, label-Depth')# Calculate microphone position in pixels mi c_x = int(mic_position[l] * self. width) mic_y = int(mic_position[0] * self.height)# Generate and overlay microphone pattern theta, response = selfgenerate_mic_pattem(pattem_type, frequency)# Scale responsefor visualization scale = min(self. height, self, width) * 0.3X = mic_x + response * scale * np.cos(theta) Y = mic_y + response * scale * np.sin(theta)# Plotmicrophone pattern ax.plot(X, Y, 'r-', linewidth=2, alpha=0.7, label=f {patem_type} pattern')#Add mic position marker ax.add_patch(Circle((mic_x,mic_y), 10, color='red'))# Add coverage zones coverage_angles = np.linspace(0, 2*np.pi, 8) for i in range(len(coverage_angles)- 1) : start_angle=coverage_angles[i] * 180 / np.pi end_angle = coverage_angles[i+l] * 180 / np.pi wedge = Wedge((mic_x, mic_y), scale, start_angle, end_angle, alpha=0. 1 , color-yellow') ax. add _patch(wedge)# Customize plot ax.set title(f Spatial Audio-Visual Overlay\n' f {pattern_type.capitalize()} Pattern at{frequency }Hz') ax.legend() ax.set_xlim(0, self. width) ax. set_ylim(self.height, 0) # Invert Y axis for image coordinates return fig def analyze_spatial_coverage(self, depth_map, mic_position, pattern_type- cardioid', frequency= 1000):. Analyze spatial coverage based on depth and mic pattern mic x = int(mic_position[l] * self. width) mic_y = int(mic_position[0] * self.height)# Generate mic pattern theta, response = self. generate_mic_pattem(pattem_type, frequency)# Create coverage map coverage_map = np.zeros like(depth map) y,x= np.ogrid[:self.height,: self, width]# Calculate angles and distancesfor each pixel angles = np.arctan2(y - mic_y, x - mic_x) distances = np. sqrt((x - mic_x)* *2 + (y - mic_y)* *2)# Interpolate mic response for each angle for i in range(self.height): forj in range(self. width):angle =angles[i,j] if angle < 0: angle +=2*np. pi idx=int(angle / (2*np.pi) *len(theta)) if idx >= len(response): idx = len(response) - 1 coverage_map[i, j ] = responsefidx]# Adjust coverage based on distance max_distance =np.sqrt(self. width* *2 + selfheight* *2) coverage_map *=(1 - distances / max_distance)# Combine with depth information coverage_map *=(l -depth_map) return coverage_map# Example usage visualizer= SpatialAudioVisualizerQ# Create visualizations for different patterns and frequencies patterns = ['cardioid', 'figure8', 'super cardioid', 'omnidirectional'] frequencies = [250, 1000, 4000] for pattern in patterns: for freq in frequencies: fig = visualizer.create_overlay_visualization( mic_position=(0.5, 0.5), pattem_type=pattem, frequency=freq) plt.show()# Analyze coverage depth map = visualizer.create_simulated_depth_map() coverage map = visualizer.analyze_spatial_coverage( depth_map, mic_position=(0.5, 0.5), pattem_type- cardioid', frequency= 1000)# Show coverage analysis plt.figure(figsize=(15, 8)) plt.imshow(coverage_map, cmap- hof) plt.colorbar(label='Coverage Intensity') pit. title(' SpatialCoverage Analysis') plt.show()

[0405] Although the invention is illustrated and described herein as embodied in one or more specific examples, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit ofthe invention and within the scope and range of equivalents of the claims. Accordingly, it is appropriate that the appended claims be construed broadly, and in a manner consistent with the scope of the invention, as set forth in the following claims.

Claims

CL IMSWhat is claimed is1. A method for rendering two different types of datasets, said method comprising: obtaining a first type of dataset, using a first sensor disposed on an AR / VR headset and that measures a first attribute at one or more three-dimensional coordinates defining a region or location in real space, and wherein said first type of dataset includes a first attribute value and an associated said three-dimensional coordinates where said first attribute value is obtained; obtaining a second type of dataset, using a second sensor that couples to said AR / VR headset that operates within a range of frequency that includes one or more frequency blocks and measures for at least one of said frequency blocks, intensity values of one or more different intangible attributes at one or more of said three-dimensional coordinates in said real space, and wherein a point of origin of said second sensor is at an offset distance relative to said point of origin of said first sensor; performing, for said frequency block, a fast Fourier transformation on said intensity values to produce a frequency domain dataset that includes one or more frequency peak values and said three-dimensional coordinates of said frequency peak values; identifying, based on one or more said frequency peak values and without using said first type of dataset, one or more intangible attribute signals inside said real space; spatializing, using a plurality of said three-dimensional coordinates, said first type of dataset and said offset distance to create a first type of spatialized dataset and a spatialized offset distance; spatializing, using said plurality of said three-dimensional coordinates, said second type of dataset to create a second type of spatialized dataset; aligning, using said spatialized offset distance, said first type of spatialized dataset with said second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and rendering, in a virtual space in said AR / VR headset and using a rendering engine, said enhanced three-dimensional spatialized environment, wherein one or more of said intangible attribute signals, associated with one or more of said frequency peak values, are visually represented in said enhanced three-dimensional spatialized environment.

2. The method for rendering two different types of datasets of claim 1, wherein said performing includes producing said frequency domain dataset using a single frequency peak having a single frequency peak value, and wherein said identifying includes ascertaining whether a frequency peak intensity value associated with said single frequency peak value equals or exceeds a predetermined intensity threshold value assigned to said frequency block.

3. The method for rendering two different types of datasets of claim 2, wherein said single frequency peak is one frequency peak of multiple frequency peaks that represent a spatial and / or temporal frequency peak pattern of a known intangible attribute signal.

4. The method for rendering two different types of datasets of claim 1, wherein said performing includes producing said frequency domain dataset using multiple temporal frequency peaks, each having a temporal frequency peak value, and / or multiple spatial frequency peaks, each having a spatial frequency peak value, and wherein said identifying includes matching, within a predefined tolerance, two or more of said multiple frequency peaks with a spatial reference pattern generated by an intangible attribute signal and / or a temporal reference pattern generated by said intangible attribute signal to identify said intangible attribute signal.

5. The method for rendering two different types of datasets of claim 4, wherein said spatial reference pattern includes multiple reference spatial frequency peaks, each of said reference spatial peaks having a reference spatial frequency value; wherein said temporal reference pattern includes multiple reference temporal peaks, each of said reference temporal peaks having a reference temporal frequency value; and wherein said matching includes matching, within said predefined tolerance, two or more of said multiple spatial frequency peak values with two or more of said reference spatial frequency values associated with said spatial reference pattern and / or multiple temporal frequency peaks values with one or more of said reference temporal frequency values associated with said temporal reference pattern.

6. The method for rendering two different types of datasets of claim 1, wherein said second type of sensor is a software defined radio (“SDR”) sensor and said identifying includes identifying multiple intangible attribute signals present in real space, wherein said rendering said enhanced three-dimensional spatialized environment further includes: visually representing multiple of said intangible attribute signals in said enhanced three- dimensional spatialized environment, wherein said intangible attribute signals are presented to be selectable; receiving selection one of said intangible attribute signals by a user; and rendering, in said virtual space in said AR / VR headset, using said rendering engine, and within said enhanced three-dimensional spatialized environment, said selected intangible attribute signal at spatialized locations that correspond to said three-dimensional coordinates of one or more of said frequency peak values that are associated with said selected intangible attribute signal.

7. The method for rendering two different types of datasets of claim 6, wherein said rendering includes presenting said intensity of said selected intangible attribute signal at one or more of said spatialized locations.

8. The method for rendering two different types of datasets of claim 7, wherein said rendering said enhanced three-dimensional spatialized environment further includes: visually representing a signal intensity of said selected intangible attribute signal with a virtual object at one or more locations in said enhanced three-dimensional spatialized environment; receiving a selection, from a user, of said virtual object at one of said locations in said enhanced three-dimensional spatialized environment; and virtually representing, within said enhanced three-dimensional spatialized environment, an intensity value of said selected virtual object.

9. The method for rendering two different types of datasets of claim 7, wherein said frequency domain dataset includes, for each of one or more of said frequency peak values, an associated frequency peak intensity value, and wherein said signal intensity of said intangibleattribute signal is said frequency peak intensity value of one of said frequency peak associated with said intangible attribute signal.

10. The method for rendering two different types of datasets of claim 8, wherein said virtual object is a virtual bar, and a size of said virtual bar corresponds to said frequency peak intensity value of one of said frequency peaks associated with said intangible attribute signal.

11. The method for rendering two different types of datasets of claim 1, wherein said second sensor is physically attached to said AR / VR headset.

12. The method for rendering two different types of datasets of claim 11, wherein said first sensor and said second sensor are disposed on multiple AR / VR headsets.

13. The method for rendering two different types of datasets of claim 12, further comprising displacing said multiple AR / VR headsets within said real space to obtain said first type of dataset and said second type of dataset, each of said AR / VR headsets measuring, using said first sensor, said first attribute value at one or more of said three-dimensional coordinates and, using said second sensor, said intensity values at one or more of said three-dimensional coordinates.

14. The method for rendering two different types of datasets of claim 1, wherein said first attribute value is a three-dimensional pixelated data value or three-dimensional voxelated data value, wherein each of said associated three-dimensional coordinates being associated with at least one of said associated three-dimensional pixelated data values or at least one of said associated three-dimensional voxelated data values.

15. The method for rendering two different types of datasets of claim 1, wherein said obtaining said second type of dataset includes removing, from said second type of dataset, one or more of said frequency blocks that do not have one or more of said measured intensity values within said frequency block.

16. The method for rendering two different types of datasets of claim 1, wherein said performing, for said frequency block, said fast Fourier transformation on said intensity valuescomprises retaining said frequency peak values that are equal to or above a peak threshold value to arrive at said frequency domain dataset.

17. The method for rendering two different types of datasets of claim 16, wherein said performing said fast Fourier transformation on said intensity values, includes performing said fast Fourier transformation on said intensities values within a time domain and / or a spatial domain to create said frequency domain dataset.

18. The method for rendering two different types of datasets of claim 1, wherein said second sensor is at least one gas concentration sensor selected from a group comprising electrochemical sensor, infrared sensor, catalytic sensor, photoionization detection sensor, metal oxide semiconductor sensor, photoacoustic gas sensor.

19. The method for rendering two different types of datasets of claim 18, wherein in said rendering said enhanced three-dimensional spatialized environment, said gas concentration is visually represented as a color gradient.

20. The method for rendering two different types of datasets of claim 1, wherein a position sensor is coupled to said second sensor to provide said three-dimensional coordinates of said second sensor in real space and wherein said obtaining said second type of dataset comprising receiving, from said position sensor, said three-dimensional coordinates of said second sensor.

21. The method for rendering two different types of datasets of claim 20, wherein said offset distance changes with time, said obtaining said offset distance includes: identifying, using one or more position sensors, one or more of said three- dimensional coordinates of said second sensor in said real space; and determining, using a tracking module, said spatial offset between one or more of said three-dimensional coordinates of said second sensor and between one or more of said three- dimensional coordinates of said first sensor, wherein said tracking module is present on any one of said AR / VR headset, said second sensor or a client device.

22. The method for rendering two different types of datasets of claim 1, wherein said spatializing said first type of dataset is carried out using an image spatializing module residing on said AR / VR headset or on an external processor that is external to said AR / VR headset, wherein said external processor being communicatively coupled to said AR / VR headset, and said method comprising: spatially partitioning, using said image spatializing module, said real space into plurality of subdivisions, wherein said real space being defined using said plurality of three-dimensional location coordinates; and integrating said subdivisions to create a spatialized model of said real space, wherein one or more first attribute values being spatially distributed, based upon said spatialized model, to create said first type of spatialized dataset.

23. The method for rendering two different types of datasets of claim 1, wherein said spatializing said second type of dataset is carried out using an attribute spatializing module residing on an AR / VR headset or on an external processor disposed external to said AR / VR headset, wherein said external processor being communicatively coupled to said AR / VR headset, and said method comprising: spatially partitioning, using said attribute spatializing module, said real space into plurality of discrete subdivisions, wherein said real space being defined using said plurality of three-dimensional location coordinates; and integrating said discrete subdivisions to create a spatialized model of said real space, wherein one or more of said intangible attribute signals being spatially distributed, based upon said spatialized model, to create said second type of spatialized dataset.

24. The method for rendering two different types of datasets of claim 23, further comprising: identifying one or more of said discrete subdivisions that contain one or more of said three-dimensional coordinates, associated with said second dataset, within a three-dimensional boundary of said discrete subdivision; and assigning said second dataset obtained to each said discrete subdivisions.

25. The method for rendering two different types of datasets of claim 24, wherein in said assigning, said second dataset values and / or multiple of said intangible attribute signals is assigned to an entire portion of said discrete subdivision.

26. The method for rendering two different types of datasets of claim 1, wherein said obtaining said second type of dataset includes obtaining, inside said real space, magnitude of at least one type of attribute chosen from a group comprising throughput of a connectivity signal, latency of said connectivity signal, interference of said connectivity signal, volatility of said connectivity signal, stability of said connectivity signal, RF power output, EMF, atmospheric pressure, geomagnetic, hall effect, ambient light level, gas levels, smoke, sound pressure, audio harmonics, humidity, carbon dioxide emission, and temperature.

27. The method for rendering two different types of datasets of claim 1, wherein said spatializing said second type of dataset includes interpolating, using two or more of said intensity value corresponding to two or more of said three-dimensional coordinates, to compute an intermediate intensity value for a corresponding intermediate three-dimensional location coordinate that is disposed between two or more of said three-dimensional coordinates and is associated with said intermediate intensity value.

28. The method for rendering two different types of datasets of claim 27, said rendering comprises rendering an intermediate virtual object at an intermediate object location, in said enhanced three-dimensional spatialized environment.

29. The method for rendering two different types of datasets of claim 1, wherein said spatializing said second type of dataset includes extrapolating, using two or more of said intensity values obtained at two or more three-dimensional location coordinates in said real space, to compute a predicted intensity value for a corresponding exterior three-dimensional location coordinate , wherein said corresponding exterior three-dimensional location coordinate is disposed outside of said real space and positioned on a linear trajectory that intersects two or more of said three-dimensional location coordinates.

30. The method for rendering two different types of datasets of claim 29, in said rendering, comprising rendering exterior virtual objects at said corresponding exterior three-dimensional location coordinates in said enhanced three-dimensional spatialized environment.

31. The method for rendering two different types of datasets of claim 7, wherein said AR / VR headset is communicatively coupled to a hand-held controller that includes one or more controllers for controlling said rendering of said enhanced three-dimensional spatialized environment, and wherein said enhanced three-dimensional spatialized environment includes a virtual representation of said hand-held controller.

32. The method for rendering two different types of datasets of claim 31, further comprising: obtaining, from one or more position sensors located on said AR headset, three- dimensional coordinates of said hand-held controller; and removing, from said enhanced three-dimensional spatialized environment, said three- dimensional representation of said hand-held controller when said hand-held controller exceeds a predetermined distance from said AR / VR headset.

33. The method for rendering two different types of datasets of claim 31, further comprising: receiving, from said user using said hand-held controller, a selection signal indicating selection by said user of a controller of a first type associated with an intangible attribute signal, which is among a multiple of said intangible attribute signals; and wherein said rendering said enhanced three-dimensional spatialized environment includes rendering said selected intangible attribute signal and said controller of said first type in an engaging manner with said virtual representation of said hand-held controller.

34. The method for rendering two different types of datasets of claim 33, further comprising: receiving, from said user using said hand-held controller, an intensity selection signal of a controller of a second type associated with said selected intangible attribute signal; and obtaining a numerical value of said intensity of said selected intangible attribute signal; wherein said rendering said enhanced three-dimensional spatialized environment includes rendering, within said enhanced three-dimensional spatialized environment, said numerical valueof said intensity value of said selected intangible attribute signal and a visual representation of said controller of said second type in an engaging manner with said virtual representation of said hand-held controller.

35. A method for rendering gas leakage, said method comprising: obtaining a first type of dataset, using a first sensor disposed on an AR / VR headset and that measures a first attribute at one or more three-dimensional coordinates defining a region or location in real space, and wherein said first type of dataset includes a first attribute value and an associated said three-dimensional coordinates where said first attribute value is obtained; obtaining a second type of dataset, using a second sensor that couples to said AR / VR headset that operates within a range of frequency that includes one or more frequency blocks and measures for at least one of said frequency blocks, intensity values at one or more of said three- dimensional coordinates in said real space, and wherein a point of origin of said second sensor is at an offset distance relative to said point of origin of said first sensor; performing, for said frequency block, a fast Fourier transformation on said intensity values to produce a frequency domain dataset that includes one or more frequency peak values and said three-dimensional coordinates of said frequency peak values; identifying, based on one or more said frequency peak values and without using said first type of dataset, one or more gas signals associated with presence of one or more gases inside said real space; spatializing, using a plurality of said three-dimensional coordinates, said first type of dataset and said offset distance to create a first type of spatialized dataset and a spatialized offset distance; spatializing, using said plurality of said three-dimensional coordinates, said second type of dataset to create a spatialized second type of dataset; aligning, using said spatialized offset distance, said first type of spatialized dataset with said second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and rendering, in a virtual space in said AR / VR headset and using a rendering engine, said enhanced three-dimensional spatialized environment and indicating, in said enhanced three- dimensional spatialized environment, presence and concentration of one or more said gases.

36. The method for rendering gas leakage of claim 35, wherein said rendering includes rendering different types of said gases in different colors.

37. The method for rendering gas leakage of claim 35, wherein said identifying presence of one or more said gas signals includes identifying one or more of said frequency peak values that are equal to or above a predetermined gas detection threshold value assigned to each of said frequency blocks.

38. The method for rendering gas leakage of claim 37, wherein rendering includes rendering one of said gases such that a first concentration range of said gas is represented by a first color, a second concentration range of said gas is represented by a second color, and a third concentration range of said gas is represented by a third color, wherein said second concentration range is of a lower magnitude than said first concentration range and said third concentration range is lower than said first concentration range and said second concentration range.

39. The method for rendering gas leakage of claim 35, wherein presence of a particular gas is identified using a predefined spatial reference pattern and / or a temporal reference pattern that is associated with said particular gas, wherein said performing includes producing said frequency domain dataset that includes multiple temporal frequency peaks, each having a temporal frequency peak value, and / or multiple spatial frequency peaks, each having a spatial frequency peak value, and wherein said identifying includes matching, within a predefined tolerance, multiple frequency peaks with said predefined spatial reference pattern and / or said temporal reference pattern to identify presence in said real space of one or more types of said gas.

40. The method for rendering gas leakage of claim 39, wherein said spatial reference pattern includes two or more reference spatial frequency peaks, each of said reference spatial peaks having a reference spatial frequency value; wherein said temporal reference pattern includes two or more reference temporal peaks, each of said reference temporal peaks having a reference temporal frequency value; andwherein said matching includes matching, within said predefined tolerance, two or more of said multiple spatial frequency peak values with two or more of said reference spatial frequency values associated with said spatial reference pattern and / or two or more of said multiple temporal frequency peaks values with two or more of said reference temporal frequency values associated with said temporal reference pattern.

41. The method for rendering gas leakage of claim 35, wherein said second sensor measures a frequency that ranges from between about 3 terahertz and about 750 terahertz.

42. The method for rendering gas leakage of claim 35, wherein said second sensor measures a frequency that ranges from between about 119 terahertz and about 428 terahertz.

43. An extended reality system for rendering a user interface, said system comprising: a first sensor disposed on an AR / VR headset for measuring, at one or more three- dimensional coordinates that define a region or location in real space, one or more first attribute values to produce a first type of dataset, wherein said first type of dataset includes a first attribute value and an associated said three-dimensional coordinates where said first attribute value is obtained; a second sensor, coupled to said AR / VR headset for measuring, that operates within a range of frequency that includes one or more frequency blocks for measuring, for at least one of said frequency block, intensity values of one or more different intangible attributes at one or more of said three-dimensional coordinates in said real space, and wherein a point of origin of said second sensor is at an offset distance relative to said point of origin of said first sensor; a display component for displaying rendered information; a processor, which is communicatively coupled to said first sensor, said second sensor, and said display component, and said processor operative to perform the following instructions: performing, for said frequency block, a fast Fourier transformation on said intensity values to produce a frequency domain dataset that includes one or more frequency peak values and said three-dimensional coordinates of said frequency peak values; identifying, based on one or more said frequency peak values and without using said first type of dataset, one or more intangible attribute signals inside said real space;spatializing, using a plurality of said three-dimensional coordinates, said first type of dataset and said offset distance to create a first type of spatialized dataset and a spatialized offset distance; spatializing, using said plurality of said three-dimensional coordinates, said first type of dataset to create a second type of spatialized dataset; aligning, using said spatialized offset distance, said first type of spatialized dataset with said second type of spatialized dataset to create an enhanced three-dimensional spatialized environment; and rendering, in a virtual space in said AR / VR headset and using a rendering engine, said enhanced three-dimensional spatialized environment wherein one or more intangible attribute signals, associated with one or more of said frequency peak values, are visually represented in said enhanced three-dimensional spatialized environment.

44. The extended reality system for rendering a user interface of claim 43, wherein said processor and said sensor being communicatively coupled in a bi-directional manner, allowing said processor to instruct said sensor to collect additional data and discard originally collected data.

45. The extended reality system for rendering a user interface of claim 43, wherein said processor being disposed in said AR / VR headset.

46. The extended reality system for rendering a user interface of claim 43, further comprising a first type of dataset spatializing module being disposed inside said processor and configured for spatializing said first type of dataset and said offset distance to create said first type of spatialized dataset and said spatialized offset distance.

47. The extended reality system for rendering a user interface of claim 43, further comprising a second type of dataset spatializing module being disposed inside said processor and configured for spatializing said second type of dataset to create one or more of said spatialized frequency peak values.

48. The extended reality system for rendering a user interface of claim 43, further comprising an aligning module being disposed inside said processor and configured for said aligning, using said spatialized offset distance, said first type of spatialized dataset with one or more of said spatialized frequency peak values to create an enhanced three-dimensional spatialized environment.

49. The extended reality system for rendering a user interface of claim 43, wherein said rendering engine is disposed inside said processor.

50. The extended reality system for rendering a user interface of claim 43, further comprising a hand-held controller, coupled to said AR / VR headset, that includes one or more controllers for controlling said rendering of said enhanced three-dimensional spatialized environment.

51. The extended reality system for rendering a user interface of claim 50, wherein said AR / VR headset includes one or more position sensors and said hand-held controller includes a light emitting diode operable to identify, using one or more of said position sensor, three dimensional coordinates of said hand-held controller in said real space.

Images