Projection device control

Abstract

The application discloses a method of expanding the display region to which a user can interact corresponding to controlling a smaller display on a device.

Description

Projection Device ControlCROSS REFERENCE TO RELATED APPLICATIONS AND PRIORITY

[0001] The present application claims priority to and the benefit of U.S. Provisional Patent Application No. 63670656, filed 12 July 2024, the entirety of which is hereby incorporated by reference.FIELD OF THE APPLICATION

[0002] The present application relates to control of devices by interaction with projected displays.BACKGROUND

[0003] Devices have become more capable of increased functions as technology advances. Advances tend toward miniaturization, which typically has advantages in both space and energy usage. However, one of the disadvantages is the size of the user interface. For example, smart watches have increased capability but very limited usage since it is difficult to interact with the watch the same way a user interacts with a computer, which has much larger user interface. A more useful interface is needed for wearables.SUMMARY

[0004] Devices, system and methods for enhanced display are disclosed. Discussions herein use sensor data to determine location and type of drone, vehicles, and acoustic sources and provide notification.

[0005] At least one exemplary embodiment is directed to a projection device, that projects a small display on a device to a larger surface projection or hologram with which a user can interact to control the device.

[0006] The projection device can include microphones, a display, a gesture or voice control sensor referred to as a remote control sensor, a projection device to project the display on a surface or as a 2D or 3D hologram the projection device also referred to as a projection sensor.

[0007] The projection device can receive a command from a user to initiate projection modeto start the process of projecting the display onto a surface or as a hologram. The projection device can detect gestures and / or voices to control a user’s interaction with the projected display. The user interactions with he projected display can mirror control of the smaller display.

[0008] These and other features of detection enhancement systems and methods are described in the following detailed description, drawings, and appended claims.BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG. 1 is a schematic diagram of the use of embodiments of the projection devices and methods in multiple systems, wired and wireless.

[0010] FIG. 2 is a schematic diagram of a projection device using a computer system within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies or operations of the systems and methods for utilizing the detection system according to embodiments of the present disclosure.

[0011] FIG. 3 illustrates a wearable configured as a projection device;

[0012] FIG. 4 illustrates a projected display from a projection device;

[0013] FIG. 5 illustrates cursor movement on the projected display;

[0014] FIG. 6 illustrates sensor emissions to detect gesture movement;

[0015] FIG. 7 illustrates additional sensor emissions to detect gesture movement;

[0016] FIG. 8 illustrates the edges of the projected display and mapping of the gesture position in relation to the display and projected display;

[0017] FIG. 9 illustrates the projected display on a palm of a user;

[0018] FIG. 10A illustrates the projected display on desk;

[0019] FIG. 10B illustrates the projection of pixels to a surface whose normal vector is parallel to the normal vector of the display surface;

[0020] FIG. IOC illustrates the projection of pixels onto a surface whose normal vector is non parallel to the display surface normal vector;

[0021] FIG. 11 illustrates projecting a computer display;

[0022] FIG. 12 illustrates a method of control of the projected display;

[0023] FIG. 13 illustrates a method of gesture control;

[0024] FIG. 14 illustrates a method of voice control;

[0025] FIG. 1 illustrates components of a projection device;

[0026] FIG. 16 illustrates gesture control detection using infrared;

[0027] FIG. 17 illustrates gesture control detection using camera(s);

[0028] FIG. 18 illustrates gesture control detection using acoustic sensor(s);

[0029] FIG. 19 illustrates a method of identifying gestures;

[0030] FIG. 20 illustrates a method of associating a gesture with a command;

[0031] FIG. 21 illustrates methods for conditioning a microphone signal;

[0032] FIG. 22 illustrates methods for analyzing a conditioned microphone signal;

[0033] FIG. 23 illustrates additional methods for analyzing a conditioned microphone signal;

[0034] FIG. 24 illustrates the method of initiating a 2-D or 3_D projection display;

[0035] FIG. 25 illustrates the creation of an emission pixel;

[0036] FIGs 26A, 26B illustrate refracted rays based upon incident angles;

[0037] FIG. 27 illustrates FIG. 26A and FIG. 26B rotated so that the incident ray is aligned;

[0038] FIG. 28 illustrates the effect of refraction control element;

[0039] FIG. 29 illustrates exciting a nitrogen atom;

[0040] FIG. 30 illustrates emission from a de-exciting nitrogen atom

[0041] FIG. 31 illustrates the primary nitrogen emission peaks;

[0042] FIG. 32 illustrates the primary oxygen emission peaks;

[0043] FIG. 33 illustrates the creation of emission pixels; and

[0044] FIG. 34 illustrates color generally as a function of wavelength.SAMPLE TERMS

[0045] The following terms are general in nature and other phrases / word can refer to similar or the same reference numbers.

[0046] 310, 320, 330, 340, 403, 405, 407, 409, 1050, 1160 sensor emitter / receivers;

[0047] 350 display;

[0048] 390 user interface;

[0049] 420, 820, 940, 1030, 1161 projected display;

[0050] 413 display interface;

[0051] 423 projected display interface;

[0052] 603, 605,607 reflected sensor emission paths;

[0053] 609 sensor emission path;

[0054] 630, 930 user control location;

[0055] Al, Bl, projected pixels;

[0056] 1090 user viewer direction;

[0057] 2510 refraction control elements;

[0058] 2520 electromagnetic emission rays;

[0059] 2531, 2532, 2533, 2534, 2535, 2541 2551, optical channel;

[0060] 2560 general interaction region;

[0061] 2570, 3320 emission pixel;

[0062] 2571 central location of emission pixel;

[0063] 2575 pixel control layer;

[0064] 2901 Nucleus; and

[0065] 2910 electron.DETAILED DESCRIPTION OF THE INVENTION

[0066] Exemplary embodiments of projection devices, and systems and methods therefore are disclosed.

[0067] Exemplary embodiments are directed to or can be operatively used with various electronic wired or wireless devices (e.g., phones, TVs, display systems, watches, bracelets, rings, voice assistant systems, tablets, laptops other devices that seek to convey information in a visual sense to a user or viewer). The system (devices or components including a least one projection device) or projection devices can communication wired or wirelessly with various devices, examples of which are mentioned above. In all of the examples illustrated and discussed herein, any specific values should be interpreted to be illustrative only and non-limiting. Thus, other examples of the exemplary embodiments could have different values and methods of user interface.

[0068] Figures 1 and 2 describe many basic components, and wired and wireless interactions that can be used by or included in a projection device or a system including a projection device,whereas specific exemplary embodiments follow therefrom. For example, as shown in Figure 1 , a projection device 115 and methods for utilizing the projection device 115 arc disclosed. The projection device 115 can use a wired and wireless system 100 to communicate data.

[0069] The system 100 may be configured to support, but is not limited to supporting, data and content services, audio processing applications and services, audio output and / or input applications and services, applications and services for transmitting and receiving audio content, authentication applications and services, computing applications and services, cloud computing services, internet services, satellite services, telephone services, software as a service (SaaS) applications, platform-as-a-service (PaaS) applications, gaming applications and services, social media applications and services, productivity applications and services, voice-over- internet protocol (VoIP) applications and services, speech-to-text translation applications and services, interactive voice applications and services, mobile applications and services, and any other computing applications and services. The system may include a first user 101, who may utilize a first user device 102 to access data, content, and applications, or to perform a variety of other tasks and functions. As an example, the first user 101 may utilize first user device 102 to access an application (e.g. a browser or a mobile application) executing on the first user device 102 that may be utilized to access web pages, data, and content associated with the system 100. In certain embodiments, the first user 101 may be any type of user that may potentially desire to listen to audio content, such as from, but not limited to, a music playlist accessible via the first user device 102, a telephone call that the first user 101 is participating in, audio content occurring in an environment in proximity to the first user 101, any other type of audio content, or a combination thereof. For example, the first user 101 may be an individual that may be participating in a telephone call with another user, such as second user 120.

[0070] The first user device 102 utilized by the first user 101 may include a memory 103 that includes instructions, and a processor 104 that executes the instructions from the memory 103 to perform the various operations that are performed by the first user device 102. In certain embodiments, the processor 104 may be hardware, software, or a combination thereof. The first user device 102 may also include an interface 105 (e.g. screen, monitor, graphical user interface, etc.) that may enable the first user 101 to interact with various applications executing on the first user device 102, to interact with various applications executing within the system 100, and tointeract with the system 100 itself. In certain embodiments, the first user device 102 may include any number of transducers, such as, but not limited to, microphones, speakers, any type of audiobased transducer, any type of transducer, or a combination thereof. In certain embodiments, the first user device 102 may be a computer, a laptop, a tablet device, a phablet, a server, a mobile device, a smartphone, a small watch, and / or any other type of computing device. Illustratively, the first user device 102 is shown as a mobile device in Figure 1. The first user device 102 may also include a global positioning system (GPS), which may include a GPS receiver and any other necessary components for enabling GPS functionality, accelerometers, gyroscopes, sensors, and any other componentry suitable for a mobile device.

[0071] In addition to using first user device 102, the first user 101 may also utilize and / or have access to a second user device 106 and a third user device 110. As with first user device 102, the first user 101 may utilize the second and third user devices 106, 110 to transmit signals to access various online services and content. The second user device 106 may include a memory 107 that includes instructions, and a processor 108 that executes the instructions from the memory 107 to perform the various operations that are performed by the second user device 106. In certain embodiments, the processor 108 may be hardware, software, or a combination thereof. The second user device 106 may also include an interface 109 that may enable the first user 101 to interact with various applications executing on the second user device 106 and to interact with the system 100. In certain embodiments, the second user device 106 may include any number of transducers, such as, but not limited to, microphones, speakers, any type of audio-based transducer, any type of transducer, or a combination thereof. In certain embodiments, the second user device 106 may be and / or may include a computer, any type of sensor, a laptop, a set-top- box, a tablet device, a phablet, a server, a mobile device, a smartphone, a smart watch, and / or any other type of computing device. Illustratively, the second user device 102 is shown as a smart watch device in Figure 1.

[0072] The third user device 110 may include a memory 111 that includes instructions, and a processor 112 that executes the instructions from the memory 111 to perform the various operations that are performed by the third user device 110. In certain embodiments, the processor 112 may be hardware, software, or a combination thereof. The third user device 110 may also include an interface 113 that may enable the first user 101 to interact with various applicationsexecuting on the second user device 106 and to interact with the system 100. In certain embodiments, the third user device 110 may include any number of transducers, such as, but not limited to, microphones, speakers, any type of audio-based transducer, any type of transducer, or a combination thereof. In certain embodiments, the third user device 110 may be and / or may include a computer, any type of sensor, a laptop, a set-top-box, a tablet device, a phablet, a server, a mobile device, a smartphone, a smart watch, and / or any other type of computing device. The user devices can receive data from at least the projection devices.

[0073] The first, second, and / or third user devices 102, 106, 110 may belong to and / or form a communications network 116. In certain embodiments, the communications network 116 may be a local, mesh, or other network that facilitates communications among the first, second, and / or third user devices 102, 106, 110 and / or any other devices, programs, and / or networks of system 100 or outside system 100. In certain embodiments, the communications network 116 may be formed between the first, second, and third user devices 102, 106, 110 through the use of any type of wireless or other protocol and / or technology. For example, the first, second, and third user devices 102, 106, 110 may communicate with one another in the communications network 116, such as by utilizing Bluetooth Low Energy (BLE), classic Bluetooth, ZigBee, cellular’, NFC, Wi-Fi, Z-Wave, ANT+, IEEE 802.15.4, IEEE 802.22, ISAlOOa, infrared, ISM band, RFID, UWB, Wireless HD, Wireless USB, any other protocol and / or wireless technology, satellite, fiber, or any combination thereof. Notably, the communications network 116 may be configured to communicatively link with and / or communicate with any other network of the system 100 and / or outside the system 100.

[0074] The system 100 may also include a projection device 115, which the first user 101 may utilize to hear and / or audition audio content, transmit audio content, receive audio content, experience any type of content, process audio content, adjust audio content, store audio content, perform any type of operation with respect to audio content, or a combination thereof. The projection device 115 may be a communication device connected to acoustic device. The projection device 115 may include any type of component utilized for any type of acoustic sensor or other sensors such as electromagnetic (visual, infrared, microwave, radio). In certain embodiments, the projection device 115 may include any number of ambient sound microphones that may be configured to capture and / or measure ambient sounds and / or audio content occurringin an environment that the projection device 115 is present in and / or is proximate to. The microphones can be part of an enhancement mechanism such as a Helmholtz chamber, discussed herein. In certain embodiments, the ambient sound microphones may be placed at a location or locations on the projection device 115 that are conducive to capturing and measuring ambient sounds occurring in the environment. For example, the ambient sound microphones may be positioned in proximity such that the ambient sound microphones are in an optimal position to capture ambient or other sounds occurring in the environment. In certain embodiments, the projection device 115 may include any number of microphones, which may be configured to capture and / or measure sounds occurring near the environment of the first user 101 or other user wearing / carrying the projection device 115.

[0075] The projection device 115 may also include any number of transceivers, which may be configured transmit signals to and / or receive signals from any of the devices in the system 100. In certain embodiments, a transceiver of the projection device 115 may facilitate wireless connections and / or transmissions between the projection device 115 and any device in the system 100, such as, but not limited to, the first user device 102, the second user device 106, the third user device 110, the fourth user device 121, the fifth user device 125, the projection device 130, the servers 140, 145, 150, 160, and the database 155. The projection device 115 may also include any number of memories for storing content and / or instructions, processors that execute the instructions from the memories to perform the operations for the projection device 115, and / or any type integrated circuit for facilitating the operation of the projection device 115. In certain embodiments, the processors may comprise, hardware, software, or a combination of hardware and software. The projection device 115 may also include one or more speaker, which may be speakers for outputting sound to the first user 101. The speakers may output sounds obtained via the array of microphones, ambient sound microphones, any of the devices in the system 100, from a storage device of the projection device 115, or any combination thereof.

[0076] The speakers, microphones, transceivers, memories, processors, integrated circuits, may be affixed to an electronics package that includes a flexible electronics board. The projection device 115 may include an electronics packaging housing that may house the ambient sound microphones, array microphones, receivers (i.e. speakers), electronics supporting the functionality of the microphones and / or receivers, transceivers for receiving and / or transmittingsignals, power sources (e.g. batteries and the like), any circuitry facilitating the operation of the projection device 115, or any combination thereof. The electronics package including the flexible electronics board may be housed within the electronics packaging housing to form an electronics packaging unit. The projection device 115 may further include a housing, which may include receptacles, openings, and / or keyed recesses for connecting the housing to the electronics packaging housing and / or the electronics package. For example, nozzles of the electronics packaging housing may be inserted into one or more keyed recesses of the housing so as to connect and secure the housing to the electronics packaging housing. When the housing is connected to the electronics packaging housing, the combination of the housing and the electronics packaging housing may form the projection device 115. The projection device 115 may further include a cap for securing the electronics packaging housing, the earphone housing, and the electronics package together to form the projection device 115.

[0077] In addition to the first user 101, the system 100 may include a second user 120, who may utilize a fourth user device 121 to access data, content, and applications, or to perform a variety of other tasks and functions. Much like the first user 101, the second user 120 may be may be any type of user that may potentially desire to listen to audio content, such as from, but not limited to, a storage device of the fourth user device 121, a telephone call that the second user 120 is participating in, audio content occurring in an environment in proximity to the second user 120, any other type of audio content, or a combination thereof. For example, the second user 120 may be an individual that may be listening to songs stored in a playlist that resides on the fourth user device 121. Also, much like the first user 101, the second user 120 may utilize fourth user device 121 to access an application (e.g. a browser or a mobile application) executing on the fourth user device 121 that may be utilized to access web pages, data, and content associated with the system 100. The fourth user device 121 may include a memory 122 that includes instructions, and a processor 123 that executes the instructions from the memory 122 to perform the various operations that are performed by the fourth user device 121. In certain embodiments, the processor 123 may be hardware, software, or a combination thereof. The fourth user device 121 may also include an interface 124 (e.g. a screen, a monitor, a graphical user interface, etc.) that may enable the second user 120 to interact with various applications executing on the fourth user device 121, to interact with various applications executing in the system 100, and to interact withthe system 100. In certain embodiments, the fourth user device 121 may include any number of transducers, such as, but not limited to, microphones, speakers, any type of audio-based transducer, any type of transducer, or a combination thereof. In certain embodiments, the fourth user device 121 may be a computer, a laptop, a tablet device, a phablet, a server, a mobile device, a smartphone, a smart watch, and / or any other type of computing device. Illustratively, the fourth user device 121 may be a computing device in Figure 1. The fourth user device 121 may also include any of the componentry described for first user device 102, the second user device 106, and / or the third user device 110. In certain embodiments, the fourth user device 121 may also include a global positioning system (GPS), which may include a GPS receiver and any other necessary components for enabling navigational operation and detection functionality, accelerometers, gyroscopes, sensors, ranging lasers, radar, and any other componentry suitable for a computing device.

[0078] In addition to using fourth user device 121, the second user 120 may also utilize and / or have access to a fifth user device 125. As with fourth user device 121, the second user 120 may utilize the fourth and fifth user devices 121, 125 to transmit signals to access various online services and content. The fifth user device 125 may include a memory 126 that includes instructions, and a processor 127 that executes the instructions from the memory 126 to perform the various operations that are performed by the fifth user device 125. In certain embodiments, the processor 127 may be hardware, software, or a combination thereof. The fifth user device 125 may also include an interface 128 that may enable the second user 120 to interact with various applications executing on the fifth user device 125 and to interact with the system 100. In certain embodiments, the fifth user device 125 may include any number of transducers, such as, but not limited to, microphones, speakers, any type of audio-based transducer, any type of transducer, or a combination thereof. In certain embodiments, the fifth user device 125 may be and / or may include a computer, any type of sensor, a laptop, a set-top-box, a tablet device, a phablet, a server, a mobile device, a smartphone, a smart watch, and / or any other type of computing device. Illustratively, the fifth user device 125 is shown as a tablet device in Figure 1.

[0079] The fourth and fifth user devices 121, 125 may belong to and / or form a communications network 131. In certain embodiments, the communications network 131 may be a local, mesh, or other network that facilitates communications between the fourth and fifthuser devices 121 , 125, and / or any other devices, programs, and / or networks of system 100 or outside system 100. In certain embodiments, the communications network 131 may be formed between the fourth and fifth user devices 121, 125 through the use of any type of wireless or other protocol and / or technology. For example, the fourth and fifth user devices 121, 125 may communicate with one another in the communications network 116, such as by utilizing BLE, classic Bluetooth, ZigBee, cellular, NFC, Wi-Fi, Z-Wave, ANT+, IEEE 802.15.4, IEEE 802.22, ISAlOOa, infrared, ISM band, RFID, UWB, Wireless HD, Wireless USB, any other protocol and / or wireless technology, satellite, fiber, or any combination thereof. Notably, the communications network 131 may be configured to communicatively link with and / or communicate with any other network of the system 100 and / or outside the system 100.

[0080] Much like first user 101, the second user 120 may have his or her own projection device 130. The projection device 130 may be utilized by the second user 120 to hear and / or audition audio content, transmit audio content, receive audio content, experience any type of content, process audio content, adjust audio content, store audio content, perform any type of operation with respect to audio content, or a combination thereof. The projection device 130 may be an earpiece, a hearing aid, an ear monitor, an ear terminal, a behind-the-ear device, any type of acoustic device, or a combination thereof. The projection device 130 may include any type of component utilized for any type of earpiece, and may include any of the features, functionality and / or components described and / or usable with projection device 115. For example, projection device 130 may include any number of transceivers, microphones, processors, memories, housings, and any other component, or any combination thereof.

[0081] In certain embodiments, the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 may have any number of software applications and / or application services stored and / or accessible thereon. For example, the first and second user devices 102, 111 may include applications for processing audio content, applications for playing, editing, transmitting, and / or receiving audio content, streaming media applications, speech-to-text translation applications, cloud-based applications, search engine applications, natural language processing applications, database applications, algorithmic applications, phone-based applications, product-ordering applications, business applications, e- commerce applications, media streaming applications, content-based applications, databaseapplications, gaming applications, internet-based applications, browser applications, mobile applications, service-based applications, productivity applications, video applications, music applications, social media applications, presentation applications, any other type of applications, any types of application services, or a combination thereof. In certain embodiments, the software applications and services may include one or more graphical user interfaces so as to enable the first and second users 101, 120 to readily interact with the software applications. The software applications and services may also be utilized by the first and second users 101, 120 to interact with any device in the system 100, any network in the system 100 (e.g. communications networks 116, 131, 135), or any combination thereof. For example, the software applications executing on the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 may be applications for receiving data, applications for storing data, applications for auditioning, editing, storing and / or processing audio content, applications for receiving demographic and preference information, applications for transforming data, applications for executing mathematical algorithms, applications for generating and transmitting electronic messages, applications for generating and transmitting various types of content, any other type of applications, or a combination thereof. In certain embodiments, the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 may include associated telephone numbers, internet protocol addresses, device identities, or any other identifiers to uniquely identify the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 and / or the first and second users 101, 120. In certain embodiments, location information corresponding to the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 may be obtained based on the internet protocol addresses, by receiving a signal from the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130 or based on profile information corresponding to the first, second, third, fourth, and / or fifth user devices 102, 106, 110, 121, 125 and / or projection devices 115, 130.

[0082] The system 100 may also include a communications network 135. The communications network 135 may be under the control of a service provider, the first and / or second users 101, 120, any other designated user, or a combination thereof. The communications network 135 of the system 100 may be configured to link each of the devices in the system 100to one another. For example, the communications network 135 may be utilized by the first user device 102 to connect with other devices within or outside communications network 135. Additionally, the communications network 135 may be configured to transmit, generate, and receive any information and data traversing the system 100. In certain embodiments, the communications network 135 may include any number of servers, databases, or other componentry. The communications network 135 may also include and be connected to a mesh network, a local network, a cloud-computing network, an IMS network, a VoIP network, a security network, a VoLTE network, a wireless network, an Ethernet network, a satellite network, a broadband network, a cellular network, a private network, a cable network, the Internet, an internet protocol network, MPLS network, a content distribution network, any network, or any combination thereof. Illustratively, servers 140, 145, and 150 are shown as being included within communications network 135. In certain embodiments, the communications network 135 may be part of a single autonomous system that is located in a particular geographic region, or be pail of multiple autonomous systems that span several geographic regions.

[0083] Notably, the functionality of the system 100 may be supported and executed by using any combination of the servers 140, 145, 150, and 160. The servers 140, 145, and 150 may reside in communications network 135, however, in certain embodiments, the servers 140, 145, 150 may reside outside communications network 135. The servers 140, 145, and 150 may provide and serve as a server service that performs the various operations and functions provided by the system 100. In certain embodiments, the server 140 may include a memory 141 that includes instructions, and a processor 142 that executes the instructions from the memory 141 to perform various operations that are performed by the server 140. The processor 142 may be hardware, software, or a combination thereof. Similarly, the server 145 may include a memory 146 that includes instructions, and a processor 147 that executes the instructions from the memory 146 to perform the various operations that are performed by the server 145. Furthermore, the server 150 may include a memory 151 that includes instructions, and a processor 152 that executes the instructions from the memory 151 to perform the various operations that are performed by the server 150. In certain embodiments, the servers 140, 145, 150, and 160 may be network servers, routers, gateways, switches, media distribution hubs, signal transfer points, service control points, service switching points, firewalls, routers, edge devices, nodes, computers, mobiledevices, or any other suitable computing device, or any combination thereof. In certain embodiments, the servers 140, 145, 150 may be communicatively linked to the communications network 135, the communications network 116, the communications network 131, any network, any device in the system 100, any program in the system 100, or any combination thereof.

[0084] The database 155 of the system 100 may be utilized to store and relay information that traverses the system 100, cache content that traverses the system 100, store data about each of the devices in the system 100 and perform any other typical functions of a database. In certain embodiments, the database 155 may be connected to or reside within the communications network 135, the communications network 116, the communications network 131, any other network, or a combination thereof. In certain embodiments, the database 155 may serve as a central repository for any information associated with any of the devices and information associated with the system 100. Furthermore, the database 155 may include a processor and memory or be connected to a processor and memory to perform the various operation associated with the database 155. In certain embodiments, the database 155 may be connected to the projection devices 115, 130, the servers 140, 145, 150, 160, the first user device 102, the second user device 106, the third user device 110, the fourth user device 121, the fifth user device 125, any devices in the system 100, any other device, any network, or any combination thereof.

[0085] The database 155 may also store information and metadata obtained from the system 100, store metadata and other information associated with the first and second users 101, 120, store user profiles associated with the first and second users 101, 120, store device profiles associated with any device in the system 100, store communications traversing the system 100, store user preferences, store information associated with any device or signal in the system 100, store information relating to patterns of usage relating to the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125, store audio content associated with the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or navigational devices 115, 130, store audio content and / or information associated with the audio content that is captured by the microphones, store audio content and / or information associated with audio content that is captured by the microphones, store any information obtained from any of the networks in the system 100, store audio content and / or information associated with audio content that is outputted by ear canal receivers of the system 100, store any information and / or signals transmitted and / orreceived by transceivers of the system 100, store any device and / or capability specifications relating to the projection devices 115, 130, store historical and biometric data associated with the first and second users 101, 120, store information relating to the size (e.g. depth, height, width, curvatures, etc.) and / or shape of the first and / or second user's 101, 120 physicality, for example ear canals and / or ears, store information identifying and or describing any devices or users utilized with the projection devices 115, 130, store device characteristics for any of the devices in the system 100, store information relating to any devices associated with the first and second users 101, 120, store any information associated with the projection devices 115, 130, store log on sequences and / or authentication information for accessing any of the devices of the system 100, store information associated with the communications networks 116, 131, store any information generated and / or processed by the system 100, store any of the information disclosed for any of the operations and functions disclosed for the system 100 herewith, store any information traversing the system 100, or any combination thereof. Furthermore, the database 155 may be configured to process queries sent to it by any device in the system 100.

[0086] The system 100 may also include a software application, which may be configured to perform and support the operative functions of the system 100, such as the operative functions of the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130. In certain embodiments, the application may be a website, a mobile application, a software application, or a combination thereof, which may be made accessible to users utilizing one or more computing devices, such as the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130. The application of the system 100 may be accessible via an internet connection established with a browser program or other application executing on the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130, a mobile application executing on the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130, or through other suitable means. Additionally, the application may allow users and computing devices to create accounts with the application and sign-in to the created accounts with authenticating username and password log-in combinations. The application may include a custom graphical user interface that the first user 101 or second user 120 may interact with by utilizing a browser executing on the first, second, third, fourth, and fifth user devices 102, 106,110, 121 , 125 and / or the projection devices 115, 130. Tn certain embodiments, the software application may execute directly as an installed program on the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130.Computing System for Facilitating the Operation and Functionality of the System

[0087] Referring now also to Figure 2, at least a portion of the methodologies and techniques described with respect to the exemplary embodiments of the system 100 and projection device system / device 115 can incorporate a machine, such as, but not limited to, computer system 14100, or other computing device within which a set of instructions, when executed, may cause the machine to perform any one or more of the methodologies or functions discussed above. The machine may be configured to facilitate various operations conducted by the system 100. For example, the machine may be configured to, but is not limited to, assist the system 100 by providing processing power to assist with processing loads experienced in the system 100, by providing storage capacity for storing instructions or data traversing the system 100, by providing functionality and / or programs for facilitating the operative functionality of the projection devices 115, 130, and / or the first, second, third, fourth, and fifth user devices 102, 106, 110, 121, 125 and / or the projection devices 115, 130, by providing functionality and / or programs for facilitating operation of any of the components of the projection devices 115, 130 (e.g. ear canal receivers, transceivers, car canal microphones, ambient sound microphones, or by assisting with any other operations conducted by or within the system 100.

[0088] In some embodiments, the machine may operate as a standalone device. In some embodiments, the machine may be connected (e.g., using communications network 135, the communications network 116, the communications network 131, another network, or a combination thereof) to and assist with operations performed by other machines and systems, such as, but not limited to, the first user device 102, the second user device 111, the third user device 110, the fourth user device 121, the fifth user device 125, the navigational device 115, the navigational device 130, the server 140, the server 150, the database 155, the server 160, or any combination thereof. The machine may be connected with any component in the system 100. In a networked deployment, the machine may operate in the capacity of a server or a client user machine in a server-client user network environment, or as a peer machine in a peer-to-peer (ordistributed) network environment. The machine may comprise a server computer, a client user computer, a personal computer (PC), a tablet PC, a laptop computer, a desktop computer, a control system, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein.

[0089] The computer system 14100 may include a processor 14102 (e.g., a central processing unit (CPU, e.g., Ml Ultra Chip Apple Chip), a graphics processing unit (GPU, or both, e.g. Nvidia's Blackwell B200 GPU and GB200 "superchip"), a main memory 14104 and a static memory 14106, which communicate with each other via a bus 14108. The computer system 14100 may further include a video display unit 14110, which may be, but is not limited to, a liquid crystal display (LCD), a flat panel, a solid-state display, or a cathode ray tube (CRT). The computer system 14100 may include an input device 14112, such as, but not limited to, a keyboard, a cursor control device 14114, such as, but not limited to, a mouse, a disk drive unit 14116, a signal generation device 14118, such as, but not limited to, a speaker or remote control, and a network interface device 14120.

[0090] The disk drive unit 14116 may include a machine -readable medium 14122 on which is stored one or more sets of instructions 14124, such as, but not limited to, software embodying any one or more of the methodologies or functions described herein, including those methods illustrated above. The instructions 14124 may also reside, completely or at least partially, within the main memory 14104, the static memory 14106, or within the processor 14102, or a combination thereof, during execution thereof by the computer system 14100. The main memory 14104 and the processor 14102 also may constitute machine-readable media.

[0091] Dedicated hardware implementations including, but not limited to, application specific integrated circuits, programmable logic arrays and other hardware devices can likewise be constructed to implement the methods described herein. Applications that may include the apparatus and systems of various embodiments broadly include a variety of electronic and computer systems. Some embodiments implement functions in two or more specific interconnected hardware modules or devices with related control and data signals communicatedbetween and through the modules, or as portions of an application-specific integrated circuit. Thus, the example system is applicable to software, firmware, and hardware implementations.

[0092] In accordance with various embodiments of the present disclosure, the methods described herein are intended for operation as software programs running on a computer processor. Furthermore, software implementations can include, but not limited to, distributed processing or component / object distributed processing, parallel processing, or virtual machine processing can also be constructed to implement the methods described herein.

[0093] The present disclosure contemplates a machine-readable medium 14122 containing instructions 14124 so that a device connected to the communications network 135, the communications network 116, the communications network 131, another network, or a combination thereof, can send or receive voice, video or data, and communicate over the communications network 135, the communications network 116, the communications network 131, another network, or a combination thereof, using the instructions. The instructions 14124 may further be transmitted or received over the communications network 135, another network, or a combination thereof, via the network interface device 14120.

[0094] While the machine-readable medium 14122 is shown in an example embodiment to be a single medium, the term "machine-readable medium" should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store the one or more sets of instructions. The term "machine-readable medium" shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that causes the machine to perform any one or more of the methodologies of the present disclosure.

[0095] The terms "machine-readable medium," "machine-readable device," or "computer- readable device" shall accordingly be taken to include, but not be limited to: memory devices, solid-state memories such as a memory card or other package that houses one or more read-only (non-volatile) memories, random access memories, or other re-writable (volatile) memories; magneto-optical or optical medium such as a disk or tape; or other self-contained information archive or set of archives is considered a distribution medium equivalent to a tangible storage medium. The "machine-readable medium," "machine-readable device," or "computer-readable device" may be non- transitory, and, in certain embodiments, may not include a wave or signalper se. Accordingly, the disclosure is considered to include any one or more of a machine- readable medium or a distribution medium, as listed herein and including art-recognized equivalents and successor media, in which the software implementations herein are stored.

[0096] The illustrations of arrangements described herein are intended to provide a general understanding of the structure of various embodiments, and they are not intended to serve as a complete description of all the elements and features of apparatus and systems that might make use of the structures described herein. Other arrangements may be utilized and derived therefrom, such that structural and logical substitutions and changes may be made without departing from the scope of this disclosure. Figures are also merely representational and may not be drawn to scale. Certain proportions thereof may be exaggerated, while others may be minimized. Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense.

[0097] Thus, although specific arrangements have been illustrated and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose may be substituted for the specific arrangement shown. This disclosure is intended to cover any and all adaptations or variations of various embodiments and arrangements of the invention. Combinations of the above arrangements, and other arrangements not specifically described herein, will be apparent to those of skill in the art upon reviewing the above description. Therefore, it is intended that the disclosure not be limited to the particular arrangement(s) disclosed as the best mode contemplated for carrying out this invention, but that the invention will include all embodiments and arrangements falling within the scope of the appended claims.

[0098] FIG. 3 illustrates a wearable 300 configured as a projection device. A wearable can be any device that can be carried (e.g., backpack, laptop, phone) and worn (e.g., ring, bracelet, watch, phone). Note that additional exemplary embodiments can be large systems that can not be carried but can be projection devices as well, such as large desktop computer systems. With reference to FIG. 3, a watch 300 can include a wrist band 360 with an adjustable band 370 with adjustable prongs and / or holes 380. The wearable 300 includes at least one sensor 310 (e.g., 310, 320, 330, 340) that can detect gestures or control motions that interact with a projected display. The sensors 310, may be separate from the projection system. For example, display 350 can be constructed, for example as discussed later (e.g., FIG. 25), to project a display (e.g.,onto an object or holographically). The sensors 310 can also act as projectors. In general sensors 310, 320, 330 and 340 can be positions on the comers of a display 350 and include emitting and receiving surfaces 393, 392, 391, and 394 respectively. The sensors (e.g. 310) can be of various types, for non-limiting examples, a camera projector, ultrasonic, and infrared.

[0099] FIG. 4 illustrates a projected display 420 from a projection device 400. The projected display distance can be chosen ahead of time and the projector designed to operate at the display distance within a threshold variation. In such a situation the accuracy of a gesture detection and control system can be enhanced by focusing on gesture detection within a threshold distance about the projected display distance. In general, a hologram (e.g., FIG. 8 and FIG. 25) will likely be designed for a specific display distance from the wearable, whereas a projected display onto a surface (e.g., FIG. 9 and FIG. 11) can have varying projected display distance. As illustrated in FIG. 4 the display 410 is emitted as a projected display 420. Labels 412 are projected into the projected display 420 as projected labels 422. Images 411 in the display 410 can be projected as projected images 421 into the projected display 420. User software controls 413 can additionally be projected as projected user controls 423 into the projected display 420. A user 430 can interact with the projected user control 423, for example press the projected play control 431, to control the wearable device 400. Determining whether a user 430 is interacting (e.g., finger motion, gestures) with the projected display 420 can be determined by the sensors (e.g., 403, 405, 407, 409). For example, a finger tip motion can be detected and associated with controlling the display 410. For example, if sensors 403, 405, 407, and 409 are ultrasonic sensors, they can project a pulse and detect the reflections to determine finger location.

[0100] FIG. 5 illustrates cursor movement 501 on the projected display 420. Gesture control from the wearable 400 can detect a user interaction with the projected display 420 and relate such control to actual control on the display 410 of the wearable 400. For example, a user can move 501 a gesture (e.g., move a finger) from point A to point B and can be mapped to the actual display 410 to move 511 a cursor from Al to Bl. Thus, a user’s interaction with the projected display is directly mapped to control of the display 410 and the wearable 400 itself.

[0101] FIG. 6 and FIG. 7 illustrates the use of emitted pulses to detect a finger location 630of a user 430. Fort example sensor 609 sends a pulse at a known time. For example, a 20KHz signal, roughly for 1 wavelength (c.g., about 1.7mm) can be sent 609, reflections 603, 605, 607 received by sensors 403, 405, and 407 respectively. The reflections 603, 605, and 607 have associated range (e.g. pathlengths) equations, R603, R605, and R607 expressed as:C(^405—£409)c(t407 C409)

[0105] The unknown is the location of the gesture of interest 630, xg, yg, and zg. The other parts of the range equations , R603, R605 and R607 are know, for example where c is the speed of sound, Xw403 is the x location of sensor 403, yW403 is the y location of sensor 403, zW403 is the z location of sensor 403, t403 is the time detected upon receipt of the reflection 603, and t409 is the time of pulse emission from sensor 409. Likewsie for the other receiving sensors. Thus, the three unknowns (e.g., xg, yg, and zg) can be solved using the three equations (R603, R605, and R607).

[0106] Verification of the solution of, xg, yg, and zg. can be accomplished by having a different sensor 407 send a pulse 707, of which reflections 703, 705 and 709 can be detected by sensors 403, 405 and 409 respectively, generating ranging equations R703, R705, and R709, which can be expressed as:C(f405—t407)C(t409 £407)

[0110] FIG. 8 illustrates the edges of the projected display 820 and mapping of the gesture position f(Xf,Yf,Zf) in the projected display 820 to the display 810. The corners of the projected display A(Xa,Ya,Za), B(Xb,Yb,Zb), C(Xc,Yc,Zc), and D(Xd,Yd,Zd) are determined by sensors a, b, c, d respectively and define the projection display plane. The finger / gesture position f(Xf,Yf,Zf) is mapped to a display cursor p(Xp, Yp, Zp), so that as the finger is moved the cursor moves. A gesture detection system can focus on gesture detection within the comers a distance 830 within a threshold distance Dth.

[0111] FIG. 9 illustrates, 900, a wearable 920 projecting a display 915 forming a projected display 971 on a palm, piece of paper, or glove 940 of a user 910. The user’s other hand 935 can be mapped for gestures to interact with the projected display 971. The corners of the projected display can be monitored and mapped 951, 961, 971, 981 by sensors 950, 960, 970, and 980 respectively. The sensors 950, 960, 970, and 980, in addition to gesture detection, can be used to project the display itself. A gesture (e.g., finger tip 930) can be detected and associated with a cursor 991 on display 915, allowing the user 935 to interact with software on display 915.

[0112] FIG. 10A illustrates a wearable 1010 with display 1020, projecting the display 1020 onto a surface 1000, for example a desk surface 1000. The projected display 1030 might be on a surface 1000 that is inclined with respect to the display 1020. The projection will have to be amended so that a user will view the projected display 1030 as if the user was looking directly down upon the display 1020, so that letters and images in the display arc viewed with the same proportions on the projected display 1030.

[0113] FIG. 10B illustrates the projection of pixels (al, bl) to a surface 1060 whose normal vector 1071 is parallel (e.g., a~0, a~180 deg) to the normal vector 1081 of the display surface (e.g., 1020). A projection device 1050 showing projecting two pixels (al, bl) from a display to another surface 1060, forming a projected display. 1090 represents a viewers perspective when viewing the surface 1060. The viewers perspective 1090 should see the pixels projected (1011, 1013) in a similar form as the original display pixel al, bl. For example, if display pixels al, bl are square, then the projected pixels 1011, 1013 should also be nearly square. To accomplish this the projection device 1050 much detect the projected surface 1060 normalvector 1081 angle a with respect to the display normal vector 1071 and adjust the projected pixels 1011, 1013, so that the shapes viewed are similar from the display (e.g., 1020) to the projected display 1060. The projected pixels (1011, 1013) are defined by the intersection of a pyramidal conical defined by edge vectors (e.g., 1051, 1071, 1053) projected on surface 1060.

[0114] FIG. 10C illustrates the projection of pixels onto a surface whose normal vector is non parallel to the display surface normal vector. If the projected pixels are not adjusted then display pixel al is mapped to inclined projected pixel 1015. However, for a viewer 1090 to see the projected display 1060 as if viewing display (e.g., 1020) the projected pixels need to be projected taking into account any angle a. Thus, projected pixel 1015 must be adjusted prior to projection so that 1015 includes 1011 and a portion of 1013, where the remainder 1017 of the second pixel is projected, so that a viewer views the projected display in FIG. 10C as if the projected display was oriented as in FIG. 10B.

[0115] FIG. 11 illustrates 1100 projecting a computer display. The computer 1110 display 1120 is emitted as a projected display 1180. A user 1150 can interact with the projected display 1180 as if the user is directly interacting with the display 1120. To do this, sensors and emitters emit the display 1120 as a projected display 1180 upon a chosen surface, in this case a wall of a room. The chosen surface can be chosen by initiating a command with a projection device to monitor gestures and then set a finger upon the surface to project a display. The gesture monitoring system can then scan the environment and determine the surface that intersects with the position of the finger. The intersecting surface then is defined as the surface to which the display 1120 will be projected. Items on the display 1120, such as images 1140 can be projected as projected images 1 170. Sensors (e.g., 1160) can monitor the gestures of the user 1150, for example the position of a finger 1190 which is then associated with a cursor 1130 on display 1120. For example, a user 1150 can move their finger to a virtual button on the projected display 1180, which moves the cursor 1130 on the display 1120 controlling the display and associated software. The sensors (e.g., 1160) can detect 1162 the edges 1161 of the projected display 1180.

[0116] FIG. 12 illustrates a method of control of the projected display. First a user can initiate projection mode 1200, which initiates the projection device to start a process of scanning an environment, preparing a projection, projecting and controlling the projecteddisplay as if controlling the original display on the wearable. The process includes determining the type of control 1205, which can include a local control of the projected display 1210, such as through the built-in wearable controls (e.g., voice control, buttons, touchscreen). Control of the projected display can also be remote 1220, for example via voice 1230 and gesture control 1270. The voice control 1230 includes several operations such as receiving a microphone signal 1235 and optionally conditioning the microphone signal 1240 (e.g., temporal filtering, spectral filtering). The original microphone signal 1235 or the conditioned microphone signal 1240 can be analyzed 1245 to detect if a user of the projected display is speaking. If the user is not speaking (NO) then the microphone signal is continued to be monitored until the user speaking is detected. Once the user voice is detected (YES) then the conditioned or original microphone signals are used to detect a voice command 1250. If NO voice command is detected from the user, then the microphone signal 1235 and / or conditioned microphone signal 1240 are monitored again for voice detection. Once a voice command is detected 1250 (YES) then the voice command is converted into a command signal 1255 to control the projection device. The command signal 1255 is sent to the processor of the projection device 1260. The other method of control can be via gestures and monitored via gesture control 1270. To detect the gesture of the user the environment is scanned 1275 to detect / identify that there is a gesture 1280. An identified gesture can then be monitored 1285 for gestures that can control a projection device. If a control gesture is recognized 1290 then the gesture can initiate control of the projection device.

[0117] FIG. 13 illustrates more specifics of gesture control 1270. The environment is scanned 1275 which can be accomplished by several methods, non-limiting examples include infrared scanning 1310, cameras 1320 and acoustic scanning 1330, the data used to construct a 3D model. The scanned environment can include detecting a user’s gesture intended for controlling a projected display. Gestures can be identified 1280 using data from camera data 1340 or other data that can be extracted from the constructed 3D model of the environment. Once a gesture is identified 1280, then the gesture is monitored 1285 to determine if the gestures are control gestures. The control gestures are then matched to gesture commands 1350 which are then used to initiate control of the projection device 1290.

[0118] FIG. 14 illustrates a method of voice control 1230. A microphone signal is received1235 from a microphone 1400. The received microphone signal 1235 can be conditioned 1240. There arc many methods of conditioning as known by one of ordincary skill in the arts of signal processing, and several non-limiting examples are filtering 1410, spectrum determination and manipulation 1420 and spectrograms 1430. The conditioned microphone signal 1240 or even the microphone signal 1235 can be analyzed to detect whether a voice exists and further whether the voice is that of a user of the projected display, although a default could be any voice. There are several methods of voice detection, several non-limiting examples can include using thresholds 1440 to compare levels, or spectrums, or math functions against.Additionally, the spectrum can be matched to detect voices, 1450. The spectrograms can also be matched to detect voices, 1460 as can level detection 1470 and match function results such as coherence and / or correlation 1480. Once a voice has been detected then the same techniques (e.g., 1440, 1450, 1460, 1470 and 1480) can be used to detect whether there is a voice command 1250. If a voice command is detected and identified then the voice command 1255 can be converted into a command signal 1255 which can then be sent to the processor of the projection device 1260.

[0119] FIG. 15 is a schematic diagram of components of a projection device 1500. A projection device can include a processor(s) 1520 to control many functions including projection, control, communication, data storage to form a detection device. The processor 1520 (e.g,, STM32 processors, audio DSP processors TMS320C6x, AV10, Altitude 16) can be connected to various memory systems, RAM 1550 (e.g., Corsair Vengeance RGB 32GB of DDR5 RAM), ROM 1560 (e.g., EEPROM), and Cache memory 1590. Either or all memory systems can store instructions and data. The processor or processors 1520 can be connected to an interface 1580 such as a keyboard, voice command system, touch display, display, holographic interface, gesture interface and other forms of interface known by hose of ordinary skill. The processor(s) 1520 can also accept audio input 1540 such as audio content signals from a communication device or remote server, or a microphone array, or other signal from memory. The projection device 1500 can be powered 1530 (e.g., battery, wired socket, solar-) or use also energy harvesting 1555 (e.g., energy harvesting Helmholtz chamber). The device 1500 can also include speakers 1565, and microphones such as an ambient sound microphone(s) 1585. Data can be transmitter from the processor(s) 1520 via a wireless audio transceiver 1570 to portable or remote device 1535 (e.g.,remote servers), which can be connected to interfaces 1545 and / or displays(s) 1525. When the detection device 1500 detects an object a notification can be sent 1510, which can be visual, acoustic or just code. Note that additional sensors 1585 can provide information to and accept instructions from the processor(s) 1520.

[0120] FIG. 16 illustrates gesture control detection using infrared or other electromagnetic imaging generating an image. Two examples are discussed but should not be taken as being the only examples that provide gesture information. The first example is using an infrared image 1600, which is also applicable for any electromagnetic imaging system. An emitter 1610, such as a infrared light source, emits waves 1615 which illuminate 1617 an object 1630, for example a user’s hand. The illumination wave 1617 bounces off 1629 the object 1630 back to 1627 a detector 1620 which detects the reflected waves 1625. The reflected waves 1625 form an image 1640 in the detector 1620 (e.g., CCD camera). This image 1640 can then be compared with stored images to detect a gesture. The second example using a pulse emission 1661 to illuminate 1667 an object, additionally results in reflected 1677 waves 1681, 1683 and 1685 to construct an image over time, but the formation of the image over time provides useful information. For example, since the finger is closer to the detector the reflected waves associated with the fingertip stimulate the CCD cells 1691 in the camera first 1692. As the pulse reflects off of other parts of the hand the image of the hand gradually forms (1694, 1696) in the CCD cells 1693 and 1695 and the time in which the cells of the CCD are stimulated provides information as to which part of the hand is closer to the detector. This information can be used to aid in the construction of a 3D model (1770) of the hand to extract gesture information.

[0121] FIG. 17 illustrates gesture control detection using camera(s) 1320. Multiple cameras 1700 can be used to drive 3D information of an environment. For example cameras 1710 and 1720 can optionally emit waves 1715 and receive waves 1725 constructing images 1750 and 1760 respectively. The reflected waves 1717 and 1727 travel to each camera 1710 and 1720 respectively to form images 1750 and 1760. Because of the cameras 1710 and 1720 different perspective with relation to the object 1730 different images are created 1750 and 1760. With the assumption of the images being from the same object, the system can then create a 3D model of the object that would create the images 1750 and 1760. This informationcan be used, optionally along with other information, to construct a 3D model 1770 of the environment, which can then later be analyzed to detect gestures.

[0122] FIG. 18 illustrates scanning an environment using acoustic sensor(s) 1800. Each acoustic sensor 1801 and 1807 can generate images such as 1640, but can also determine distance information to aid in a 3D model. For example, each sensor 1801, 1807, can emit waves or pulses 1813 and which travel 1815, 1825 to an object 1830 in the environment. The waves can be reflected off of parts of the object 1830, for example a finger 1829. The finger can be at a location (x,y,z). The reflected waves / pulses 1823 travel back 1817, 1827 to the sensors 1801, 1807. In addition to images such as 1640 the acoustic sensors can generate range (distance) information 1803 and 1805 as expressed in equations (1) through (6).

[0123] FIG. 19 illustrates a method of identifying gestures by matching images to stored images associated with gestures 1920. Images 1640, formed by cameras, sensors or other methods such as projecting the 3D model of 1770 onto a 2D plane, can be compared to stored images of gestures to be on the look out for 1900. The image 1640 can be manipulated, for example rotated 1910, then compared 1930 to images stored 1900. When a manipulated image is matched to a stored image 1930, then the gesture can be associated with a command.

[0124] FIG. 20 illustrates a method of associating a gesture with a command 1350. Images that are identified as gestures can then be compared to specific images that are associated with commands. The command gesture images 2000 can be associated with particular commands 2001 to control the projection device (e.g., wearable). For example image 2010 can be associated with the command stop 2011, image 2020 associated with he command to lower a level 2021, image 2030 associated with he command to take control of a cursor and move it to another location 2031, image 2040 can be associated with the command of end of action 2041, and image 2050 can be associated with he command to follow the identified TIP in the image to move a cursor 2051. Note than image 1640 most closely matches a command associated with image 2050.

[0125] In addition to detecting gesture commands, voice interaction can be used for remote control, and example of the steps are illustrated in Fig. 14. FIGs. 21-23 illustrates non-limiting examples of the steps in FIG. 14 with respect to conditioning a microphone signal. A nonlimiting example of applying 2110 a filter 1410 is shown in FIG. 21. A filter 2111 can beapplied to a microphone signal 2113 received from a microphone 1400 generating a conditioned signal 2115. A non-limiting example of applying determining a spectrum 1420 from a microphones signal 2121 is shown in FIG. 21. A math function (e.g., FFT) can be applied to the microphone signal 2121 to generate a spectrum 2123. A filter can also be applied to the generated spectrum to generate 2130 a filtered spectrum 1411. For example, a spectrum filter 2131can be applied to a spectrum 2133 to generate a filtered spectrum 2135. Additionally, spectrograms 1430 can be generated by spectrums 2133 or filtered spectrum 2135. A spectrogram is composed of spectrums associated with particular sample time spans. For example, spectrum 2141 is associated with temporal microphone’s data from time t to t+dt. Likewise, spectrums 2143, 21445, 2147 and 2149 are associated with their own time spans. These spectrums can be combined to form 2150 spectrogram 2160.

[0126] FIG. 22 illustrates methods for analyzing a conditioned microphone signal using math functions 1440 (e.g., coherence, correlation) and level detections 1450. For example a correlation can be used to compare the microphone or conditioned microphone signal to a target signal 2215 or a separate part of the microphone signal to generate a correlation value V 2217. The value can be against a threshold to detect a voice. Other non-limiting examples such as level detection 1450 can be used to detect voices, for example various level examples 2200 are shown in FIG. 22. Various levels 2210, 2220, 2230 and 2240 can be compared against a microphone signal 2113, a conditioned microphone signal 2115, a spectrum 2133,and a filtered spectrum 2135 respectively. Likewise, non linear levels 2250, 2260, 2270, and 2280 can be compared against a microphone signal 2113, a conditioned microphone signal 2115, a spectrum 2133, and a filtered spectrum 2135 respectively.

[0127] FIG. 23 illustrates additional methods for analyzing a conditioned microphone signal to detect and identify voice commands, using non-limiting examples such as thresholds 1440, spectrum matching 1450 and spectrogram matching 1460. FIG. 23 illustrates examples of thresholds 2310, 2320 and 2330 used in comparison of a microphone signal 2113, an amplitude of a filtered spectrum 2135, and a frequency range of a filtered spectrum 2135 respectively. Spectrum matching can also be used to detect and identify voices. For example, 2123 shows a range of spectrum amplitude values 2350, as function of frequency, associated with a known voice can be compared to the spectrum 2351 to determine if a voice is detected.It can also be used to identify a voice command. Likewise, spectrogram matching 1460 can be used to detect not only voices but actual words forming a voice command. For example, a constructed spectrogram 2160 from measured microphone signals can be spectrogram matched 2370 to stored values 2360 to determine if voices exists and to determine the actual words and whether they are voice commands. For example, spectrogram 2160 matches stored spectrogram 2380, and thus the generated spectrogram can be associated with the voice command of ‘‘play”. Thus, a voice can be detected and a command identified.

[0128] FIG. 24 illustrates the method of initiating a 2-D or 3_D projection display. Projection 2410 can be simply projecting the display by using known projector technology. The hologram 2420 projected display can be 2D 2430 or 3D 2440. In the 2D hologram projection a projection area 2432 is defined 2431, whereas in the 3D hologram projection a projection volume 2431 is defined 2471. In the 2D projection an array of emission pixels 2438, created by emitters 2433, 2437 and 2435 are arranged in the 2D defined plane 2432. In the 3D projection a matrix of emission pixels 2438, created by emitters 2433, 2437 and 2435 are arranged in the 3D defined region 2431. The emitter rays 2435A, 2437A , 2433A are designed to intersect in region 2436 creating an emission pixel 2438, which emits light 2434.

[0129] FIG. 25 illustrates the creation of an emission pixel, additionally described in more detail in FIG. 33. An emission pixel 2570, a pixel that emits light from a location away from the display of a wearable, can be generated by interference and superposition of multiple rays intersecting withing a region of 2560 of which the emission pixel 2570 is a subset region. The intersecting rays 2537 and 2543 can angled, with respect to a normal vector 2590 of an emitting surface such that they intersect in a region 2560 associated with a desired location 2571 (X, Y, Z). The angle of the rays can be controlled by a refraction element 2510 overlay ed upon an optical channel layer 2575. Rays 2520 and 2540 passing through their respective optical channels impact the refraction element 2510, each seeing a different curvature that bends the rays upon leaving the refraction element 2510 at different angles. To have specific rays intersect at a desired location 2571 then the rays can be chosen to be emitted from optical channels that will, in combination with the refraction element 2510 intersect at the location, for example channels 2599 are associated with refraction curvatures that intersect at the location.Each refractive sub element 2511 can be associated with several optical channels 2593. Different optical channels, 2531, 2531, 2533, 2534, 2535 will bend the ray 2520 is a different direction upon exiting the refractive element 2510. Thus a display can be projected by sending different rays of different frequencies along channels that will allow the rays to intersect at a desired location for the emission pixel that those rays will create. Thus, a fine enough (i.e., more optical channels than emission pixels) will allow a multicolor 2D and #d projection at designed distances. The emission device that creates the 2D or 3D hologram projection can be a separ ate device or part of the display of the wearable. For example some parts of the refractive element can be designed not to refract 2553 the rays 2550, so that a viewer could view the emission device as the display on the wearable itself.

[0130] FIGs 26 A, 26B illustrate refracted rays based upon incident angles. A refraction control element 2510 can include surface and index of refraction variations so that projection rays from different optical channels can cross at a predetermined distance from the display to create the emission pixel forming the projected display, for example a hologram. Figures 26A and 26B illustrate methods to manipulate a projection ray 2600 and 2620 to various refracted directions 2610 and 2630 respectively dependent upon the index of refraction (e.g., nl, n2) difference’s in accordance with Snell’s Law and incident angles. For example, if nl=1.6 for optical plastic and n2=1.0 for air, then al=30deg, and bl=35deg, results in a2=53.1deg, and b2=66.6deg respectively.

[0131] FIG. 27 illustrates FIG. 26A and FIG. 26B rotated so that the incident ray is aligned simulating the rays passing through similar optical channels. To determine the shape needed for the refraction control element 2510, the projection rays 2600 and 2620 can be overlapped in FIG. 26A and 26B to illustrate that a surface incline so as to varying surface normal vectors 2605, 2625 can result in refracted directions 2610, 2630 that are different. This is a nonlimiting example of how to control where the projection rays intersect. An example of varying surface in a refraction control element 2510 is shown in FIG. 28, which combines the refracted rays 2610 and 2630 shown in FIG. 27.

[0132] FIG. 29 illustrates exciting a nitrogen atom. Creating an emission pixel requires enticing an air molecule to excite and then de-excite, emitting light. FIG. 29 illustrates 2900 the process of excitation of a nitrogen molecule, of which air is mostly composed along withoxygen. In general, a nucleus 2901 is surrounded by electrons (2910, 2950) at various orbitals 2920, 2930 and 2940. To excite an electron 2950, a photon 2960 having energy equal to the energy gap between orbitals, between 2940 and 2930, can be absorbed moving electron 2950 to a higher energy state (excited) 2980. That excited electron 2980 can later lose its energy (deexcited) emitting a photon of energy equal to the difference in energies between the orbitals it started and ended at. For example, FIG. 30 illustrates emission from a de-exciting nitrogen atom. An excited electron 3080 is de-excited 3070 from orbital 2940 to orbital 2930. The deexcited electron 3050 rests now in orbital 2940. When the electron moves from 2940 to 2930 a photon is emitted whose energy equals the energy difference in the orbital energies. An atom or molecule with many electrons and orbitals can have many energy differences between orbits and thus can have many different photon emitted energies from excited to de-excited states. Photon energy is directly related to wavelength, since it is assumed that the speed of light does not change, and the wavelengths are directly related to color. For example, FIG. 34 illustrates various colors and their related wavelength in nanometers.

[0133] FIG. 31 illustrates the primary nitrogen photon emission peaks (bright lines). Thus, excited Nitrogen can emit colors from blue, green and red. FIG. 32 illustrates the primary oxygen photon emission peaks (bright lines) and likewise has blue, red and green emission lines. For any optical color, a combination of red, green and blue can generate any other color. Thus to create an emission pixel, a pixel in space that emits color, a combination of de-excited photons can combine to form a major color emitted by the emission pixel.

[0134] FIG. 33 illustrates a non-limiting example of the creation of emission pixels. Electromagnetic waves (rays) of certain frequencies are emitted, 2543, 2553, 2537. The frequencies are such that they do not exactly match the absorption frequencies close to the desired emission frequencies of the emission pixel. This is so that the rays can pass through the air without undue absorption until they overlap at the desired region 3320. The rays 2543, 2553 and 2537 overlap in a larger general region 2571 within which 3315 the rays overlap and undergo interference and superposition. The desired region 3320 is where the overlap of the rays results in frequencies that can excite the air (e.g., Nitrogen and Oxygen), even though the original rays do not have the exact frequencies needed for excitation. For example, if the spectrum of rays is measured outside of region 2571 the spectrum 3310 would be obtained.However, within the region 3320, the overlap of the waves results in a spectrum measure of 3330, in which several additional frequencies peaks appeal’ due to the interference and superposition of the rays. The ray frequencies can be chosen so that new frequencies peaks fnewi and fnew2 are at the desired absorption frequencies. These absorbed frequencies excite electrons in Nitrogen and Oxygen which triggers emissions of color that appear from the region 3320 creating an emission pixel as the excited electron lose energy during de-exciting. A projection device, in addition to just projecting the display, can optionally or alternatively create emission pixels related to the display or a 3D model creating projected display that emits color from a chosen location. This color emitted display, either 2D or 3D is also referred to herein as a hologram, and can optionally be interacted with by a user.

[0135] The foregoing is provided for purposes of illustrating, explaining, and describing embodiments of this invention. Modifications and adaptations to these embodiments will be apparent to those skilled in the art and may be made without departing from the scope or spirit of this invention. Upon reviewing the aforementioned embodiments, it would be evident to an artisan with ordinary skill in the art that said embodiments can be modified, reduced, or enhanced without departing from the scope and spirit of the claims described below.

Claims

CLAIMSI claim:

1. A projection device comprising: a microphone; a display configured to show a portion of display data; a remote control sensor; a projection sensor; a memory that stores instructions; and a processor, operatively connected to the microphone, operatively connected to the remote control sensor, operatively connected to the projection sensor, where the processor is operatively connected to the memory, where the processor is configured to execute the instructions to perform operations, the operations comprising: receiving an initiate projection mode signal; sending a scan command to the remote control sensor; receiving scan data from the remote control sensor; receiving a projection command; sending a project display signal to the projection sensor; receiving a ready signal from the projection sensor; retrieving the display data; generating projection display data from the display data; sending the projection display data to the projection sensor, wherein the projection sensor projects the projection display data as a projected display; analyzing the scan data for a remote control command from a user; generating a command signal from the remote control command; and enacting a control associated with the command signal.

2. The projection device according to claim 1 wherein the projection device is a wearable.

3. The projection device according to claim 2, wherein the wearable is a phone or a tablet or a laptop.

4. The projection device according to claim 2, wherein the display is an LED display.

5. The projection device according to claim 2, wherein the remote control sensor is a camera.

6. The projection device according to claim 2, wherein the remote control sensor is an acoustic sensor.

7. The projection device according to claim 5, wherein the camera is an infrared camera.

8. The projection device according to claim 2, wherein the projection sensor is a camera or video projector.

9. The projection device according to claim 2, wherein the projection sensor is an emission surface, wherein the emission surface includes a refraction element and an optical channel layer.

10. The projection device according to claim 8, wherein the scan data is image data.

11. The projection device according to claim 6, wherein the scan data is acoustic data.

12. The projection device according to claim 1, wherein the projection command is to project a surface projection.

13. The projection device according to claim 1, wherein the projection command is to project a hologram.

14. The projection device according to claim 13, wherein the command is to project a 3D hologram.

15. The projection device according to claim 1, wherein the display data can be data for a 2D surface projection, or a 2D hologram, or a 3D hologram.

16. The projection device according to claim 11, wherein the remote control command is a voice command.

17. The projection device according to claim 1, wherein the remote control command is a hand gesture.

18. The projection device according to claim 16, wherein the voice command is identified by matching the spectrum of the scan data with stored spectrums of voice commands.

19. The projection device according to claim 17, wherein the hand gesture is identified and associated with a remote control command by matching a gesture image with stored command gesture images.

20. The projection device according to claim 1, wherein the control is to move a cursor on the display.

Images