Apparatus, kits, media, methods, and systems for composing haptic tracks and energy prescriptions for one or more users

Abstract

Aspects of the disclosure are directed to a haptic track composition system. The system may provide a haptic instrument creating haptic tracks for use in entertainment, therapy, medicine, communication, security, education, etc. Haptic tracks may include music or musical notes encoded with temporal spatial information mapped to each haptic feedback. Each haptic feedback may include different energy types and magnitudes. The system may transform the haptic tracks into haptic outputs and cause one or more multi-energy generators that are attached to or placed adjacent to the skins of a user' body to output haptic signals. The tool may also be utilized to build a library of haptic tracks. The stored haptic tracks may be used to train a machine-learning model to build customized libraries of the haptic tracks for each user.

Description

APPARATUS, KITS, MEDIA, METHODS, AND SYSTEMS FOR COMPOSING HAPTIC TRACKS AND ENERGY PRESCRIPTIONS FOR ONE OR MORE USERSCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 619,800 filed January 11, 2024, and U.S. Provisional Patent Application No. 63 / 703,121 filed October 3, 2024, each of which is incorporated by reference herein in its entirety.TECHNICAL FIELD

[0002] Aspects of the present disclosure related to apparatus, kits, media, methods, and systems for composing haptic tracks and energy prescriptions. Particular aspects relate to enabling real-time composition for one or more users.BACKGROUND

[0003] Haptic technology involves outputting one or more different types of energy toward the skin to stimulate nerves associated with the skin and / or underlying tissues to realize a user experience, such as simulating a sense of touch. Different output levels may be used to realize different user experiences. For example, one set of magnitudes may be used in communication applications, such as for entertainment by enhancing user experiences when using game controllers, joysticks, or steering wheels. As a further example, a similar and / or different set of magnitudes may be used in therapeutic and wellness applications, such as for mental therapy by stimulating the parasympathetic nervous system and / or for physical therapy by affecting tissues underneath the skin. A wide range of single-energy haptic technologies have been developed to enable different types of user experiences when positioned on, adjacent to their skin, or under the skin. Multi-energy haptic technologies may utilize a plurality of different energy types, such as electricity, thermal (hot or cold), optical (e.g., red light), pressure (e.g., direct, sonic, or ultrasonic), vibration (e.g., including impact and / or oscillating forces), etc., making them superior for many applications in communications and / or therapy.BRIEF SUMMARY

[0004] Aspects of the disclosure are directed to a haptic track composition and energy prescription system. The haptic track composition and energy prescription system may provide a tool for creating long-form haptic tracks for use in entertainment, therapy, communication,security, education, medicine, etc. Haptic tracks (described interchangeably as “energy prescriptions” or “energy tracks” in this disclosure) may be paired with music, audio, or any other time-based media (e.g., a silent film of dance, a movie, etc.). Each haptic feedback may include one or more of a plurality of different energy types (e.g., such as electricity, thermal (hot or cold), optical (e.g., red light), pressure (e.g., direct, sonic, or ultrasonic), vibration (e.g., including impact and / or oscillating forces), etc.) output at different magnitudes (e.g., in terms of frequency, intensity, and / or amplitude) depending on the application. The haptic track composition and energy prescription system may allow users to import audio-based files or data and transform the files into haptic outputs to cause one or more multi-energy generators that are attached to or placed adjacent to the skins of the body to output haptic signals. The multi-energy generators may be worn at different locations on the body. The tool may also be utilized to build, share, and sell a library of haptic tracks. The stored haptic tracks may be used to train a machine-learning model to build customized libraries of the haptic tracks for each user based on their specifications, making the described tools mass-customizable.

[0005] Aspects of the disclosure provide for apparatus, kits, media (e.g., non-transitory computer-readable media), methods, and / or systems for generating haptic tracks and energy prescriptions, comprising: receiving, by one or more processors, input data; determining, by the one or more processors, control signals from the received input data; and causing, by the one or more processors, one or more energy generators to output energies responsive to the control signals.

[0006] In an example, the method may comprise receiving modifying data from a user.

[0007] In yet another example, the method may comprise causing the one or more energy generators to output modified energies responsive to the modifying data.

[0008] In yet another example, the method may comprise storing the modifying data to train a machine learning model.

[0009] In yet another example, the input data includes MIDI data, MP3 data, temporal- spatial data related to musical notes, haptic layout information related to types of hardware, associated 3D layout and interconnection, or data related to mapping the musical notes to haptic signals.

[0010] In yet another example, the control signals include pressure, heat, light, electrical stimulation, or vibration.

[0011] In yet another example, the control signals are tailored for entertainment, therapy, education, security or medicine.

[0012] Aspects of the disclosure are directed to a system including a first generator comprising first generating elements that are independently operable to output a first plurality of different energy types; a second generator comprising second generating elements that are independently operable to output a second plurality of different energy types; an instrument configured to receive user input representing a desired energy output and generate input signals in response to receiving the user input, the instrument including a plurality of data inputs defining a first subset and a second subset, the first subset being mapped to the first generating elements of the first generator and the second subset mapped to the second generating elements of the second generator; and a control unit operably connected to the instrument, the control unit configured to analyze the input signals and generate an energy track by encoding the input signals into a musical instrument digital interface format. The energy track may be operable to cause the first generator and the second generator to output a coordinated stimulus by causing the first generating elements to output the first plurality of different energy types and the second generating elements to output the second plurality of different energy types.

[0013] In an example, the instrument includes an electric piano keyboard and the plurality of data inputs comprise keys on the electric piano keyboard. In another example, the first and second subsets comprise a first and second octave of keys, respectively, on the electric piano keyboard.

[0014] In another example, the first and second generators are configured to output a plurality of energy types including a first energy type and a second energy type, wherein the electric piano keyboard includes a modulation wheel moveable from a first position to a second position, wherein the input signals cause the first and second generators to output the first energy type when the modulation wheel is in the first position, and wherein the input signals cause the first and second generators to output the second energy type when the modulation wheel is in the second position.

[0015] In another example, a velocity of a keystroke may be mapped to energy output by at least one of the first and second generators. In another example, a storage medium isconfigured to store the energy track for playback, wherein the storage medium may be configured to synchronize the energy track with an audio track. In another example, the energy track may be generated in real time as the user input is received. In another example, the instrument outputs musical notes and generates the input signals simultaneously. In another example, the control unit may be configured to map the musical notes to energy signals. In another example, the control unit further may include an interface enabling a user to modify the generated energy track. In another example, the machine-readable format may include a time-sequenced series of instructions interpretable by the first and second generators. In another example, the first and second generators are configured to output at least one of pressure, heat, light, electrical stimulation, sound, or vibration.

[0016] Aspects of the disclosure are directed to a system and method of composing a haptic track, comprising capturing user input with a haptic instrument having a plurality of data inputs defining a first subset and a second subset, the first subset mapped to a first haptic generator and the second subset mapped to a second haptic generator; processing the captured user input to extract haptic event data; converting the haptic event data into a musical instrument digital interface format; and outputting the musical instrument digital interface format as a haptic track.

[0017] In an example, the haptic instrument may include an electric piano keyboard, the plurality of data inputs comprise keys on the electric piano keyboard, and capturing user input may include capturing a keystroke by the user. In another example, the plurality of data inputs may include a plurality of keys defining the first subset of data inputs and a second plurality of keys defining the second subset of data inputs. In another example, the method may comprise synchronizing the haptic track with an audio track. In another example, the haptic event data may include timing, intensity, and frequency of the haptic event.

[0018] In another example, the method may comprise outputting the haptic track to the first and second haptic generators thereby causing the first and second haptic generators to output haptic energy. In another example, the output haptic energy of the first and second haptic generators may comprise one or more of a plurality of different energy types (e.g., such as electricity, thermal (hot or cold), optical (e.g., red light), pressure (e.g., direct, sonic, or ultrasonic), vibration (e.g., including impact and / or oscillating forces), etc.) output at different magnitudes (e.g., in terms of frequency, intensity, and / or amplitude) depending on theapplication. In another example, the method may comprise receiving user input on the haptic instrument to change the output haptic energy type of at least one of the first and second haptic generators.

[0019] Aspects of the disclosure are directed to a system. For example, the system may comprise a first haptic generator comprising first generator elements configured to output a first plurality of different haptic energies on a first side of a user’s body; a second haptic generator comprising second generator elements configured to output a second plurality of different haptic energies on a second side of a user’s body; and a haptic controller operable to cause the first haptic generator and the second haptic generator to output a coordinated bilateral stimulus to the first and second sides of the user’s body by generating and transmitting a first haptic control signal and a second haptic control signal to the first and second haptic generators, respectively, so that the coordinated bilateral stimulus communicates a frequency difference between the first and second pluralities of different haptic energies.

[0020] In an example, the first haptic generator may output haptic energy at a first frequency, the second haptic generator may output haptic energy at a second frequency, and the second frequency may be different from the first frequency to cause a user to perceive a third frequency at the difference between the first and second frequency. In another example, the second frequency may change by a second frequency delta in response to a change in the first frequency by a first frequency delta. In another example, the second frequency delta may be equal to the first frequency delta. In another example, the frequency differences may be about 4.99 Hz. In another example, the first haptic generator may be coupled to a user’s left wrist and the second haptic generator may be coupled to the user’s right wrist.

[0021] In an example, the first and second haptic generators may be integrated into a wearable device configured for bilateral frequency delivery. In another example, the haptic controller may include a timer configured to adjust the haptic energy output of the first and second haptic generators according to a time-based progression to guide a user through stages of relaxation, focus, or sleep. In another example, the haptic controller may be configured to analyze environmental frequency noise using an integrated sensor and adjust the first and second frequencies signals to optimize the communicated frequency difference. In another example, the third frequency may be adjusted by adjusting the first and second frequencies to achieve a desired mental or physiological outcome.

[0022] In another example, the system may comprise a haptic instrument comprising an electric piano keyboard configured to send a first and second haptic signals to the first and second haptic generators, respectively. In another example, the haptic controller may be configured to wirelessly communicate with the first and second haptic generators. In another example, the first and second plurality of different haptic energies comprise at least two of pressure, heat, light, electrical stimulation, or vibration.

[0023] Aspects of the disclosure are directed to a method of generating bilateral stimulation, comprising generating, by a haptic controller, a first haptic energy signal; generating, by the haptic controller, a second haptic energy signal at a second frequency; transmitting, by the haptic controller, the first haptic energy signal to a first haptic generator coupled to a first side of a user’s body, thereby causing the first haptic generator to generate vibration and light; transmitting, by the haptic controller, the second haptic energy signal to a second haptic generator coupled to a second side of the user’s body, thereby causing the second haptic generator to generate vibration and light, thereby providing a coordinated bilateral stimulus that communicates frequency differences in vibration and light.

[0024] In an example, the method may comprise adjusting, by the haptic controller, the second haptic energy signal to maintain a constant frequency difference between the first and second haptic generators. In another example, the first and second haptic energy outputs are further configured to output at least one of heat and electrical stimulation. In another example, the first haptic generator may be coupled to the user’s right wrist and the second haptic generator may be coupled to the user’s left wrist.

[0025] In another example, the method may comprise monitoring a physiological parameter of the user with a biometric sensor; and adjusting at least one of the first haptic energy signal and the second haptic energy signals in response to a change in the physiological parameter.

[0026] Aspects of the disclosure are directed to a haptic energy output system, comprising a plurality of haptic generators on or adjacent a plurality of audience members; a haptic instrument operable by a performer, the haptic instrument operatively coupled with the plurality of haptic generators, the haptic instrument configured to receive user input representing a desired haptic output, generate haptic signals in real time in response to receivingthe user input, and transmit the haptic signals to the plurality of haptic generators thereby causing the haptic generators to emit light and vibration.

[0027] In an example, the haptic instrument may be an electric piano keyboard. In another example, the electric piano keyboard may include a modulation wheel moveable from a first position to a second position, wherein the haptic signals cause the haptic generators to emit a first haptic energy type when the modulation wheel is in the first position, and wherein the haptic signals cause the haptic generators to emit a second haptic energy type when the modulation wheel is in the second position.

[0028] In another example, the haptic instrument outputs musical notes and generates the haptic signals simultaneously. In another example, the system may comprise an activator configured to activate or deactivate at least one of the plurality of haptic generators. In another example, the plurality of haptic generators include a first plurality of haptic generators in a first section; a second plurality of haptic generators in a second section; and a third plurality of haptic generators in a third section, wherein the activator may be configured to selectively activate or deactivate all of the haptic generators in one of the first, second, or third sections.

[0029] In another example, the plurality of haptic generators may include a first plurality of haptic generators coupled to a right wrist of the audience members and a second plurality of haptic generators coupled to a left wrist of the audience members and the haptic instrument may be operable to transmit a first haptic signal to the first plurality of haptic generators and a second haptic signal to the second plurality of haptic generators.

[0030] In another example, the first and second haptic signals provide a coordinated bilateral stimulus that communicates frequency differences in vibration and light. In another example, the haptic instrument electronically communicates with the plurality of haptic generators via at least one of message queuing telemetry transport, Bluetooth, Zigbee, Z-Wave, or near field communication protocols. In another example, the haptic instrument may be configured to sense a position of the haptic generators and transmit haptic signals to cause select ones of the plurality of haptic generators to emit light, thereby creating a selected visual effect across the audience.

[0031] In another example, a first plurality of haptic generators are respectively coupled to seats in an arena and the haptic instrument may be configured to induce movement of the audience members by causing the haptic generators coupled to the seats to vibrate. In anotherexample, the haptic generators are configured to emit at least one of heat and electrical stimulation. In another example, the system may comprise a display configured to display a virtual representation of the haptic generators and relative positions within an arena, and a status of the haptic generators.

[0032] Aspects of the disclosure are directed to a method for generating haptic energy output, comprising receiving, by a haptic instrument having a plurality of user inputs, a user selection indicating a desired haptic output; generating, by the haptic instrument, haptic signals based on the user input; transmitting, by the haptic instrument, the haptic signals to a plurality of haptic generators on or adjacent a plurality of audience members; and causing the plurality of haptic generators to emit light and vibration in response to the haptic signals.

[0033] In an example, the haptic instrument may include an electric piano keyboard including a modulation wheel, and the method may comprise transmitting a first haptic signal when the modulation wheel is in a first position, thereby causing the plurality of haptic generators to emit a first haptic energy type; and transmitting a second haptic signal when the modulation wheel is in the second position, thereby causing the plurality of haptic generators to emit a second haptic energy type.

[0034] In another example, the method may comprise simultaneously outputting musical notes and generating the haptic signals with the haptic instrument. In another example, the plurality of haptic generators are coupled to or adjacent to a plurality of users, and the method further includes inducing movement of the users by causing the haptic generators to emit haptic energy.

[0035] In another example, the method may comprise transmitting a first haptic signal to a first subset of the haptic generators; and transmitting a second haptic signal to a second subset of the haptic generators, wherein the first and second haptic signals a coordinated bilateral stimulus that communicates frequency differences in vibration and light of the first and second subsets of haptic generators.

[0036] In another example, the haptic signals are transmitted to the plurality of haptic generators via at least one of Bluetooth, Zigbee, Z-Wave, or near field communication protocols. In another example, the generating step and the transmitting step are performed in real time in response to receiving the user selection. In another example, the method may comprise activating or deactivating at least one of the plurality of haptic generators using anactivator prior to transmitting the haptic signals. In another example the plurality of haptic generators are divided into a first section, a second section, and a third section, and the method includes selectively activating or deactivating all haptic generators in one of the sections using the activator.

[0037] In another example, the method may comprise simultaneously operating the haptic instrument and the activator to adjust the haptic output of at least one of the haptic generators. In another example, the activator may include at least one of a head mounted device and a wand having onboard circuitry for indicating an orientation of the activator and the method further may include determining that the activator is pointed at a subset of the haptic generators; and activating or deactivating the subset of haptic generators. In another example, the activator includes a camera-readable indicia for determining the orientation of the activator.BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings, which are incorporated in and constitute part of this disclosure, illustrate exemplary aspects that, together with the written descriptions, serve to explain the principles of this disclosure. Numerous aspects are particularly described, pointed out, and taught in the written descriptions. Some structural and operational aspects may be even better understood by referencing the written portions together with the accompanying drawings, of which:

[0039] FIG. 1 depicts a block diagram of an example haptic device connected to a display screen according to aspects of the disclosure.

[0040] FIG. 2 depicts an example user interface environment for modifying haptic information according to aspects of the disclosure.

[0041] FIG. 3A-3B depicts a block diagram illustrating a haptic track composition and energy prescription system according to aspects of the disclosure.

[0042] FIG. 4 depicts an example wearable haptic device displayed on a user interface screen according to aspects of the disclosure.

[0043] FIG. 5 depicts an example wearable haptic device for the leg of a user according to aspects of the disclosure.

[0044] FIG. 6 depicts an example wearable haptic device for the arms of a user according to aspects of the disclosure.

[0045] FIG.7 depicts a block diagram illustrating components of a haptic track composition and energy prescription system according to aspects of the disclosure.

[0046] FIG. 8 depicts an example library system for haptic track and energy prescriptions according to aspects of the disclosure.

[0047] FIG. 9 depicts a flow diagram of an example process for generating haptic tracks and energy prescriptions according to aspects of the disclosure.

[0048] FIG. 10 depicts an example wearable haptic device for the arms or legs of a user according to aspects of the disclosure.

[0049] FIG. 11 depicts the wearable haptic device of FIG. 10 with the watchband removed.

[0050] FIG. 12 depicts the wearable haptic device of FIG. 11 with the cover removed.

[0051] FIG. 13 depicts an exploded view of the wearable haptic device of FIG. 11.

[0052] FIG. 14 depicts a bottom perspective view of the wearable haptic device of FIG. 11.

[0053] FIG. 15 depicts a bottom perspective view of the cover of the wearable haptic device of FIG. 11.

[0054] FIG. 16 depicts an example haptic device according to aspects of the disclosure.

[0055] FIG. 17 depicts a person wearing multiple haptic devices according to aspects of the disclosure.

[0056] FIG. 18 depicts an example of a group haptic composition experience according to aspects of the disclosure.

[0057] FIG. 19 depicts a top plan view of a haptic instrument according to aspects of the disclosure.

[0058] FIG. 20 depicts an example of a group haptic composition experience with a composer utilizing a haptic instrument and an activator.

[0059] Some aspects depicted in the drawings may be explained further by way of citations to their drawing and element numbers. The drawings and any citations thereto are provided for illustration purposes, and to further clarify the description of the present disclosure. They are not intended to limit the present disclosure unless claimed.DETAILED DESCRIPTION

[0060] Aspects of the present disclosure are not limited to the exemplary structural details and component arrangements described in this description and shown in the accompanying drawings. Many aspects of this disclosure may be applicable to other aspects and / or capable of being practiced or carried out in various variants of use, including the examples described herein. Any example or variation may be claimed.

[0061] Throughout the written descriptions, specific details are set forth in order to provide a more thorough understanding to persons of ordinary skill in the art. For convenience and ease of description, some well-known elements may be described conceptually to avoid unnecessarily obscuring the focus of this disclosure. In this regard, the written descriptions and accompanying drawings should be interpreted as illustrative rather than restrictive, enabling rather than limiting.

[0062] Exemplary aspects of this disclosure reference apparatus, kits, media, methods, and systems for composing haptic tracks and energy prescriptions for one or more users, in which either term may be interchangeably described as an “energy track”. Some aspects are described with reference to particular technologies (e.g., multi-energy generators) operable in particular ways (e.g., via a controller operable over a wireless network) to realize particular benefits (e.g., mass-customized experiences for one or more users) for particular users (e.g., living being, such as mammals or reptiles) under particular conditions (e.g., when co-located and / or remotely located). Unless claimed, these exemplary aspects are provided for convenience and not intended to limit this disclosure.

[0063] Several reference axes are described, including: a longitudinal axis X-X, a lateral axis Y-Y, and a vertical axis Z-Z. Various aspects are described relative to these axes. Each axis X-X, Y-Y, and Z-Z may define relative arrangements. For example, each longitudinal axis X-X may be non-parallel with at least one lateral axis Y-Y in some perspectives, meaning that axis Y-Y may extend across and / or intersect axis X-X. Terms such as “long” and “elongated” may describe any aspect having a length along one of axes X-X, Y-Y, or Z-Z that is longer in relation to a width along a non-parallel one of axes X-X, Y-Y, or Z-Z. Additional axes, movements, and forces also may be described with reference to axes X-X, Y-Y, and / or Z-Z. Anatomical terms such as “anterior” and “posterior,” “medial” and “lateral,” and “proximal” and “distal” may be used to describe some structures in relation to an exemplary position and / or orientation. These relative terms are provided for convenience and ease of description, and do not limit this disclosure unless claimed.

[0064] As used herein, inclusive terms such as “comprises,” “comprising,” “includes,” “including,” and variations thereof, are intended to cover a non-exclusive inclusion, such that any apparatus, methods, system, or element thereof described herein as comprising an exemplary list of elements does not include only those elements, but may include other elements not expressly listed and / or inherent thereto. Unless stated otherwise, the term “exemplary” is used in the sense of “example,” rather than “ideal,” and does not limit this disclosure unless claimed. Various terms of approximation may be used in this disclosure, including “approximately” and “generally.” Unless stated otherwise, approximately means within 10% of a stated number or outcome and generally means “within most cases” or greater than 50% chance.

[0065] Terms such as “attach,” “attachable with,” and “attached to” are used in this disclosure to describe connections between two or more elements. Some connections may be non-removably and / or non-rotatably attached, such as when two elements are formed together and cannot be rotated and / or separated without damage. Other connections may be removably and / or rotatably attached, such as when two elements are coupled together by engagement elements (e.g., bolts, pins, rods, screws, etc.) to form structures (e.g., joints, hinges, etc.) allowing the elements to be rotated relative to one another and / or separated from another without damage. The term “pin” is used as an exemplary means for placing one element such (e.g., a first circuit board) in slidable attachment with another (e.g., a second circuit board) and should be broadly interpreted to include any structures suitable for obtaining slidable relationships. Accordingly, unless stated otherwise, such terms and their equivalents should be broadly interpreted to comprise any obvious variations thereof.

[0066] Aspects of exemplary computing technologies are described herein with reference to terms like “controller” or “microcontroller”. Functional terms such as “processing,” “computing,” “calculating,” “determining,” “displaying,” and the like, may refer to actions and processes performable by any controller described herein, which may comprise any type of software and / or hardware. The software of the controller may comprise program objects (e.g., lines of codes) that are executable locally and / or with a remote computing technology (e.g., a cloud-based system) to perform various functions. Each program object may comprise a sequence of operations leading to a desired result, such as an algorithm. The operations may require or involve physical manipulations of physical quantities, such as electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated.The signals may be described conceptually as bits, characters, elements, numbers, symbols, terms, values, or the like.

[0067] The hardware of the controller may comprise any memory technologies for storing the program objects and any data associated therewith. For example, the program objects may be stored in any machine (e.g., a computer) readable storage medium in communication with the processing unit, including any mechanism for storing or transmitting data and information in a form readable by a machine (e.g., a computer). Exemplary storage mediums may comprise read only memory (“ROM”); random access memory (“RAM”); erasable programmable ROMs (“EPROMs”); electrically erasable programmable ROMs (“EEPROMs”); magnetic or optical cards or disks; flash memory devices; and / or any electrical, optical, acoustical, or other form of propagated signals, such as carrier waves, infrared signals, digital signals, and the like.

[0068] In keeping with above, any controller described herein may comprise and / or be operable with a smartphone or similar device, such as an iPhone or other iOS device, an Android phone or other Android device, or any comparable and / or compatible devices operable to perform any functions described herein with reference to any controller.

[0069] Some aspects of the present disclosure are described with reference to methods with steps that may be performable with the controller. To help orient the reader, some methods may be described with reference to a conceptual drawing, such as a flowchart with boxes interconnected by arrows. Each box may represent a particular step or technology. The boxes may be combined, interconnected, and / or interchanged to provide options for additional modifications according to this disclosure. The arrows may define an exemplary sequence of operation for the steps, the order of which may be important. For example, a particular order of the steps may describe a sequence of operation that is performable by the controller to realize specific processing benefits, such as improving a computational performance and / or an operational efficiency.

[0070] Aspects of this disclosure generally relates to tools for creating single or multienergy haptic tracks that allow the user to receive audio-based files and transform the files into haptic outputs. The haptic track composition and energy prescription system may configure the multi-energy generator to generate haptic signals utilizing a plurality of different energy types, such as electricity, thermal (hot or cold), optical (e.g., red light), pressure (e.g., direct, sonic, or ultrasonic), vibration (e.g., including impact and / or oscillating forces), etc., based on the transformed haptic outputs. The haptic track composition and energy prescription system mayalso provide a user interface on personal electronic devices such as a smartphone, laptop, tablet, or gamepad to display virtual representations of the multi-energy generators and portions of the human body to simulate and edit the haptic outputs.

[0071] The haptic track composition and energy prescription system may generate haptic signals and allow a user to spatially map or assign a plurality of energy generators to output long-form haptic signals. A collection of information including spatial mappings or assigning of each energy generator, energy types available for each generator, and values representing the magnitude of each type of energy may be input. Further, input information for the haptic track composition and energy prescription system may include MIDI files output from a known Digital Audio Workstation (DAW), a physical sound output made from devices such as a digital piano, computer keyboard, digital synthesizer, drum, beat pads or other digitally connected input or audio files downloaded or streaming from commercial sites or any other software input device using API, data-stream or customized input such as touch open sound control (OSC).

[0072] The haptic track composition and energy prescription system may receive the above input data, allow the mapping of the haptic output, and assign the input data to the energy type and the location of each generator. The haptic track composition and energy prescription system may also generate, load and / or save multiple haptic tracks. The users may create patterns or pre-configured haptic signal combinations, edit haptic tracks, visualize haptic tracks, and modify the multi-energy generator’s outputs using 3D models.

[0073] The haptic track composition and energy prescription system may also perform two- way communication with multi-energy generators and receive information such as the user’s skin temperature, the multi-energy generators’ temperatures, status, and the battery life of the multi-energy generators. The haptic track may change based on the received information. The haptic track composition and energy prescription system may be simultaneously run on multiple devices. The devices may include laptops, tablets, smartphones, etc.

[0074] The present disclosure is advantageous as it may provide long-form haptic signals in multiple types, individually assigned to each multi-energy generator, and may be controlled by using a controller via a graphical user interface. The present disclosure may also simultaneously output multiple types of haptic signals using a single generator.

[0075] FIG. 1 depicts a block diagram of an example haptic device connected to a display screen. Haptic device 102 may include one or more multi-energy generators. In this example, haptic device 102 may include node 104, node 106, and node 108. Without departing from this disclosure, nodes 104-108 may include aspects of the energy generators described in U.S. Patent No. 10,959,674 and is progeny, US Patent App. No. 17 / 797,361 and its progeny, US Patent No. 11,934,583 and its progeny, US Patent App. No. 17 / 922,791 and its progeny, Int’l Patent App. No. PCT / US23 / 34768 and its progeny, US Patent App. No. 18 / 879,809 and its progeny, and / or US Patent App. No. 18 / 945,391 and its progeny, the entireties of which are hereby incorporated by reference into this disclosure.

[0076] Each node may generate haptic signals. For example, node 104 may generate heat, light, vibration, pressure, or any combination of the haptic signals. Each node may generate identical or different combinations of haptic signals at a time. For example, node 104 may generate vibration and heat while node 106 and node 108 may only generate light and pressure. Haptic device 102 may also include controller 110. Controller 110 may be communicable with display screen 112 via any known wired and / or wireless networking technology. Controller 110 may monitor the status of each node. The status information may include locations, temperatures, battery life, malfunctions, etc. Controller 110 may send the status include information for display on a display screen 112. The display screen may include a graphical user interface. A user may toggle the status bar assigned to each node and each parameter. In this example, the parameter may include colors (red, orange, blue), vibration, and temperatures. Although only three nodes and a few parameters are illustrated in FIG. l, N numbers of nodes may be monitored and the status of the N numbers of nodes may be displayed. Also, any other parameters for any types of haptic signals or movements of the nodes may be controlled using display screen 112 (e.g., pressure, rolling, tapping, etc.)

[0077] Each bar assigned to each parameter and node may be toggled using a computer- readable gesture, such as by mobbing the user’ s finger in air and / or across a touchscreen. When the user changes the status of each parameter, controller 110 may simultaneously or almost simultaneously change the magnitude of the haptic signals or types of haptic signals of each node. Controller 110, for example, changes the colors that node 104 is emitting to orange while increasing the temperature of node 104. Controller 110 may store the status and changes in the status of each node in memory (not shown). The stored information may be used to train a machine learning model such that the machine learning model may automatically generate pre-set configurations of each node and store with haptic tracks. Haptic tracks may be stored with temporal-spatial information as to the status of each node. Haptic tracks may be stored in the form of a collection of musical notes or any other type of sound information in MIDI, MP3, or other known audio formats. In some examples, haptic tracks may only include time stamps and the status of each parameter relative to each node. While the data may be stored using a medium traditionally used for music, no actual musical information needs to be stored in a haptic track.

[0078] FIG. 2 depicts an example graphical user interface environment for modifying haptic signal information. The graphical user interface 202 (GUI 202) may allow a user to stop, fast-forward, play, or rewind using clicking buttons 204. GUI 202 may provide project name 206. Project name 206 may be a file name of a haptic track. Haptic tracks may include music. Waveform viewer 208 represents graphic representations of waves that indicate the frequency and amplitude of sound contained in a sound or music file that the haptic track composition and energy prescription system receives. GUI 202 may also allow a user to choose effect menu 220. The user may select and change each effect from the drop-down list.

[0079] GUI 202 may also enable a user to select automation controls. Automation controls tab 224 may allow the user to select either “frequency” or “modulation”. If the user selects “frequency”. The haptic track composition and energy prescription system may automatically change the distance between the start and end of each wave represented in waveform 208. For example, the haptic track composition and energy prescription system may retrieve historical information of the user from local or cloud-based storage and modify the frequencies of the waves accordingly. The user’s historical information may include the user’s interactions with GUI 202 in the past. If the user manually modified each frequency and / or modulation and timing and / or duration of haptic signals in the past, the haptic track composition and energy prescription system may store such information as the user’s historical information.

[0080] GUI 202 may further display a group of nodes, groups 210-218. For the illustration of this example only, each group may include four nodes. Toggle switch 222 may represent whether groups 210-218 are included in arm cuff 1 (e.g., right arm) or arm cuff 2 (e.g., left arm). GUI 202 may also indicate haptic signals using square bares such as bars 226-232. Each bar may be assigned to a specific node in each of groups 210-218. For example, bar 226 may be assigned to the first node of group 218, bar 228 may be assigned to the second node, and so on. When bars 226-232 appear, each corresponding node outputs a haptic signal. In thisexample, GUI 202 may allow the user to select a type of haptic signal from selection tab 203. If “vibration” is selected by the user, each bar may indicate the timing of the vibration that should be output by the corresponding node. In other examples, GUI 202 may enable the user to select multiple types of haptic signals. In such examples, bars in different shapes or colors may be overlay ed with one another to indicate each different haptic signal.

[0081] FIG. 3A-3B depicts a block diagram illustrating components of example haptic track composition and energy prescription system. Referring to FIG. 3 A, controller 110 described in connection with FIG.l may include various components. For example, controller 110 may include project controller 310, input controller 213, array controller 314, node controller 322, and input player 316. Input controller 312 may receive sound information from MIDI input device 302 and haptic map 306. MIDI input device 302 may include synthesizers, samplers, and computers. MIDI input device 302 may also include virtual instruments inside a digital audio workstation (DAW). Input controller 312 may also receive project information from project controller 310. Project information may include haptic project 308. Project information may include mapping information, track information, file format information, any flags or warnings recorded during the past playback of the project, grid settings for each node displayable on a GUI, nodes or group selection information, and past edit and playback information. This information may be saved in a single file format and may be retrieved by project controller 310. Input controller 312 may receive haptic map information 306. Haptic map information 306 may include mapping information of each musical note to nodes. Haptic map information 306 may also include information as to which notes trigger the output of haptic signals or which notes correspond to which types of haptic signals. As input controller 312 receives musical or sound information from MIDI input device 302, input controller 312 may send such information to haptic map information 306 to check against the previously stored notes information. Based on the comparison with the information contained in haptic map information 306, input controller 12 may send musical or sound information mapped to certain nodes / triggers / types of haptic signals to input player 316.

[0082] Input player 316 may receive information from track player 318. Track player 318 may include a graphical user interface that enables a user to control playhead, piano rolls, and visualization of the grid and sound waveform of MP3 files. Track player 318 may also display haptic tables with which the user may control or modify different parameters associated with each node. Track player 318 may receive haptic tack information 304. Haptic track information304 may include information related to MIDI data (notes, timestamps, etc.), haptic layout, haptic map, and MP3 Universal Unique Identifier (UUID). The above information may be stored as metadata with the MIDI file. Track player 318 may also receive information from track composer 320. Track composer 320 may record sound information or music using a piano or any other instrument. Track composer 320 may interact with track player 318 to control the playhead, graphic displays of MP3 files, etc. Input player 315 may send the information received from input controller 312 and track player 318 to array controller 314. Array controller 314 may receive haptic project information from project controller 310 and haptic layout information 324. Haptic layout information 324 may include information as to which node is assigned which node array and the numbers of nodes in such array. Haptic layout information may also include transform details such as position, rotation, and scale information. Haptic layout information may further include reference objects and relationships with other objects. Haptic layout information may also include additional information related to haptic signals, their associated colors, and any other customizations that has been implemented.

[0083] Array controller 314 may receive both haptic project information and haptic layout information and send such information to node controller 322. Node controller 322 may interact with each node and send information as to the haptic signal output (time, magnitude, type, etc.). Node controller 322 may send the above output information to node state 326. Each node may have its node state and each node’s state may be visualized and displayed using node visualization 328. Node visualization 328 may be displayed on an overlay display over track player 318. Node state 326 may be changed based on the information transmitted from node controller 322. Any changes in node state 325 may be sent to haptic device 334 as output information, node haptic 330. Haptic device 334 may include groups of nodes such as groups 210-218 as depicted in FIG.2 Haptic device 334 may initiate or modify the haptic signals that each node in each group outputs. Haptic device 334 may receive LED simulation information from node LED simulator 332. Node LED simulator 332 may simulate the lighting of each node according to node state 326. Node LED simulator 332 may translate non-LED haptic signals into an RGB color indicative of which haptic signal would be firing if not in simulator mode. Node LED simulator 332 may demonstrate the lighting of each node in advance to the user or track composer 320 such that the user or track composer 320 may determine whether the haptic output information that will be sent to haptic device 334 may comply with the haptic output information programmed or modified by the user or track composer 320.

[0084] FIG. 4 depicts an example wearable haptic device displayed on a user interface screen. GUI 402 may be equivalent to node visualization 328 as depicted in FIG. 3B. GUI 402 may display wearable haptic device 404 placed on a user’s right arm. Wearable haptic device 404 may include twelve (12) nodes. As shown in FIG. 4, for example, each node of wearable device 404 may be a self-contained apparatus that is adhered to the user’s skin with a biocompatible adhesive in a spaced apart configuration that maintains air gaps between the nodes. Node array 410 may represent each node in a different color. The enlarged node may indicate that the node is outputting haptic signals. Temperature 408 may indicate a selected node’s temperatures. GUI 402 may allow a user to change the number indicated in temperature 408 by re-typing a desired value. Similarly, frequency 406 may indicate the frequency of vibration exerted by a node from node array 410. The user may change the frequency by retyping a desired value.

[0085] FIG. 5 depicts an example wearable haptic device. Knee brace 502 may be worn by a user. Knee brace 502 may include nodes 504-510. Each node is placed on different parts of the user’s leg. Each node may be controlled independently and simultaneously by using GUI 402 or node visualization 328. The haptic track composition and energy prescription system may configure each node to output the haptic signals received by node controller 322 based on haptic project information 308, haptic layout information 324, haptic map information 306, and haptic track information 304. The haptic track composition and energy prescription system may also manipulate the configuration of each node using a graphical user interface while the nodes are outputting the above haptic signals as the user modifies input information by input controller 312 or track player 318 modified by track composer 320.

[0086] FIG. 6 depicts an example wearable haptic device. A user may wear a node 602 attached to a cuff wrapped around the user’s right arm and node 604 attached to the cuff wrapped around the user’s left arm. The user may wear nodes 602 and 604 while playing video games or viewing media content such as a TV show or movie. Each node will trigger and output haptic signals when the controller is connected to nodes 602 and 604 using a microphone or application connected to the sound components of the TV or other types of media display screen and the sound system detects certain notes or semantics mapped to particular haptic signals. For example, if the controller detects an exploding sound, node 602 may be triggered and output haptic signals such as heat and pressure. When the controller detects a certain word mapped to a certain haptic signal from a dialogue between two or more characters appearingin a movie, the controller may trigger either node 602 or 604 to output the pre-mapped haptic signal.

[0087] FIG. 7 depicts a block diagram illustrating components of a haptic track composition and energy prescription system. User computing device 712 may include a television, mobile computing device, or other user device with a display that may be adapted to GUI 202 or GUI 402. Server computing device 715 may be a component of the haptic track composition and energy prescription system illustrated in FIGS.3A-3B. Server computing device 715 may provide haptic project information 308, haptic layout information 324, haptic map information 306, etc., to the user computing device 712 for display. Server computing device 715 may further provide historical information of each user, such as how the user previously changed the configuration of each node or whether the user finished a haptic track from start to end without quitting. User computing device 712 may transmit a request from a user to modify the parameters associated with each type of haptic signals. Storage devices 730 may store further information regarding the user or states of nodes. In response to receiving the request, server computing device 715 may retrieve further information from storage devices 730.

[0088] User computing device 712 and the server computing device 715 may be communicatively coupled to one or more storage devices 730 over a network 760. The storage device(s) 730 may be a combination of volatile and non-volatile memory and may be at the same or different physical locations than the computing devices 712, 715. For example, the storage device(s) 730 may include any type of non-transitory computer-readable medium capable of storing information, such as a hard-drive, solid state drive, tape drive, optical storage, memory card, ROM, RAM, DVD, CD-ROM, write-capable, and read-only memories.

[0089] The server computing device 715 may include one or more processors 713 and memory 714. Memory 714 may store information accessible by the processor(s) 713, including instructions 721 that may be executed by the processor(s) 713. Memory 714 may also include data 723 that may be retrieved, manipulated, or stored by the processor(s) 713. Memory 714 may be a type of non-transitory computer-readable medium capable of storing information accessible by the processor(s) 713, such as volatile and non-volatile memory. The processor(s) 713 may include one or more central processing units (CPUs), graphic processing units(GPUs), field-programmable gate arrays (FPGAs), and / or application-specific integrated circuits (ASICs), such as tensor processing units (TPUs).

[0090] Instructions 721 may include one or more instructions that when executed by the processor(s) 713, cause the one or more processors to perform actions defined by the instructions. Instructions 721 may be stored in object code format for direct processing by the processor(s) 713, or in other formats including interpretable scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. Instructions 721 may include instructions for implementing processes consistent with aspects of this disclosure. Such processes may be executed using the processor(s) 713, and / or using other processors remotely located from the server computing device 715.

[0091] The data 723 may be retrieved, stored, or modified by the processor(s) 713 in accordance with instructions 721. Data 723 may be stored in computer registers, in a relational or non-relational database as a table having a plurality of different fields and records, or as JSON, YAML, proto, or XML documents. Data 723 may also be formatted in a computer- readable format such as, but not limited to, binary values, ASCII, or Unicode. Moreover, data 723 may include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories, including other network locations, or information that is used by a function to calculate relevant data.

[0092] ML model 722 may be stored in memory 714. ML model 722 may be a machine learning model trained using various information such as settings and configuration of different nodes in the past, information related to haptic tracks played by the user in the past, and any changes made by the user while playing back the haptic tracks. ML model 722 may be continually trained each time a different haptic track is played such that ML model 722 may automatically generate a haptic track customized to a particular user.

[0093] User computing device 712 may also be configured similar to the server computing device 715, with one or more processors 716, memory 717, instructions 718, and data 719. The user computing device 712 may also include a user output 726, and a user input 724. The user input 724 may include any appropriate mechanism or technique for receiving input from a user, such as a keyboard, mouse, mechanical actuators, soft actuators, touchscreens, microphones, and sensors.

[0094] Server computing device 715 may be configured to transmit data to the user computing device 712, and the user computing device 712 may be configured to display at least a portion of the received data on a display implemented as part of the user output 726. The user output 726 may also be used for displaying an interface between the user computing device 712 and the server computing device 715. The user output 726 may alternatively or additionally include one or more speakers, transducers or other audio outputs, a haptic interface or other tactile feedback that provides non-visual and non-audible information to the platform user of the user computing device 712.

[0095] Although FIG. 7 illustrates the processors 713, 716 and the memories 714, 717 as being within the computing devices 715, 712, components described in this specification, including the processors 713, 716 and the memories 714, 717 may include multiple processors and memories that may operate in different physical locations and not within the same computing device. For example, some of the instructions 721, 718 and the data 723, 719 may be stored on a removable SD card and others within a read-only computer chip. Some or all of the instructions and data may be stored in a location physically remote from, yet still accessible by, the processors 713, 716. Similarly, processors 713, 716 may include a collection of processors that may perform concurrent and / or sequential operations. Computing devices 715, 712 may each include one or more internal clocks providing timing information, which may be used for time measurement for operations and programs run by computing devices 715, 712.

[0096] The server computing device 715 may be configured to receive requests to process data from the user computing device 712. For example, environment 700 may be part of a computing platform configured to provide a variety of services to users, through various user interfaces and / or APIs exposing the platform services.

[0097] Devices 712, 715 may be capable of direct and indirect communication over network 760. Devices 712, 715 may set up listening sockets that may accept an initiating connection for sending and receiving information. The network 760 itself may include various configurations and protocols including the Internet, World Wide Web, intranets, virtual private networks, wide area networks, local networks, and private networks using communication protocols proprietary to one or more companies. Network 760 may support a variety of short- and long-range connections. The network 760, in addition, or alternatively, may also supportwired connections between devices 712, 715, including over various types of Ethernet connection.

[0098] Although a single server computing device 715 and user computing device 712 are shown in FIG. 7, it is understood that the aspects of the disclosure may be implemented according to a variety of different configurations and quantities of computing devices, including in paradigms for sequential or parallel processing, or over a distributed network of multiple devices. In some implementations, aspects of the disclosure may be performed on a single device, and any combination thereof.

[0099] FIG. 8 depicts an example library system for haptic track composition and energy prescriptions. Library system 800 may include user television equipment 802, user computer equipment 804, wireless user communication device 806, haptic track library 816, and haptic track library 818 are coupled to communication network 824 via paths 808, 810, 812, 820, and 822. User television equipment 802, user computer equipment 804, and wireless user communication device 806 may comprise one or more user computing devices 712 as illustrated in FIG.7. Haptic track library 816 and haptic track library 818 may comprise one or more server computing devices 715 as illustrated in FIG.7. User computer equipment 804 and user television equipment 802 may be Internet-enabled and allow for access to Internet content. User computer equipment 804 may include a graphical user interface that allows a user to select a particular haptic track from either haptic track library 816 or 818. Haptic track library 816 may only contain free haptic tracks that users may download at no cost. Haptic track library 818 may provide premium haptic tracks that the users must pay to download. Downloaded haptic tracks may be tailored to the display capabilities of both user computer equipment 804 and user television equipment 802. For example, on user computer equipment 804, the haptic tracks may be provided via a website accessed by a web browser. Paths 808, 810, 812, 820, and 822 may include one or more communication paths, such as a satellite path, a fiber-optic path, a cable path, a path that supports internet communication or any other suitable wired or wireless communication path or a combination of such paths.

[0100] Haptic track library 816 may include one or more types of content distribution equipment including a television distribution facility, cable system headend, satellite distribution facility, programming source, intermediate distribution facilities and / or server, internet providers, on-demand media server, and other content providers. Haptic track library818 may also include cable sources, satellite providers, on-demand providers, Internet providers, over-the-top content providers, or other providers of content. Haptic track libraries 816 and 818 may include an interactive program guide that receives haptic track guide data via a data feed. Haptic track libraries 816 and 818 may also include user data. For example, the user data may include current and / or historical user activity information such as what haptic tracks the user typically downloads or streams, what times of day the user downloads or streams the haptic track, etc. The haptic track guide data may further include subscription data. The subscription data may identify to which sources or providers of haptic tracks a given user subscribes and / or to which sources or providers the given user has previously subscribed but later terminates access.

[0101] FIG. 9 depicts a flow diagram of an example process for generating haptic tracks and energy prescriptions. According to block 902, the haptic track composition and energy prescription system may receive input data. The input data may include pre-composed music files stored with haptic overlay information. Haptic overlay information may include which note corresponds to a trigger for a haptic signal. The haptic overlay information may also include the time of the start and the end of the triggering note or signal.

[0102] According to block 904, the haptic track composition and energy prescription system may determine control signals from the received input data. The haptic track composition and energy prescription system may determine when to trigger one or more energy generators and what type of energy that each generator should output. The haptic track composition and energy prescription system may determine such information from, for example, haptic project information and haptic layout information as described in connection with FIG. 3 A.

[0103] According to block 906, the haptic track composition and energy prescription system may cause one or more energy generators to output energies to the skin of a user responsive to the control signal. The one or more energy generators may include the nodes as depicted in FIG. 2 and FIG. 4. Each energy generator may output different types of energy at different magnitudes according to the determined control signals. For communications and / or mental health applications, each energy generator may output different types of energy at a first, lower magnitude for stimulating nerves associated with the skin; whereas, for physical health applications, each energy generator may output different types of energy at a second,higher magnitude for deadening nerves associated with the skin (e.g., for pain management) and / or for affecting physiological tissues and structures below the skin (e.g., for relaxing muscles and / or healing bones). Either way, each haptic track may contain sequences of repeated notes at different intervals; and each energy generator may be triggered at different times and for varying duration in accordance with the content of the haptic track.

[0104] According to block 908, the haptic track composition and energy prescription system may receive modifying data from the user. The user may modify the control signal while the haptic track is being played. For example, the user may utilize a graphical user interface on the user device to change parameters such as temperatures or magnitudes of the pressure output by individual energy generators.

[0105] According to block 910, the haptic track composition and energy prescription system may cause the one or more energy generators to output modified energies responsive to the modifying data. The haptic track composition and energy prescription system may adjust the parameters associated with each haptic signal that each energy generator was outputting as the system receives the modifying data from the user via the graphical user interface.

[0106] According to block 912, the haptic track composition and energy prescription system may store the modifying data. The haptic track composition and energy prescription system may store the modifying data in local or cloud-based storage. The modifying data may include the timestamp information, which parameters were modified, whether frequency or amplitude were modified, which generator relative to the user’s body part was modified, etc. The above information may be stored and used to train a machine-learning model.

[0107] FIG. 10 depicts an example haptic device 1002. Haptic device 1002 is a wearable haptic device that may be easily put on and taken off by a user. Haptic device 1002 includes one or more energy generators configured to emit different energy types (heat, light, pressure, vibration, electrical stimulation, etc.). Like haptic device 102 described above, without departing from this disclosure, haptic device 1002 also may include aspects of the energy generators described in U.S. Patent No. 10,959,674 and is progeny, US Patent App. No. 17 / 797,361 and its progeny, US Patent No. 11,934,583 and its progeny, US Patent App. No. 17 / 922,791 and its progeny, IntT Patent App. No. PCT / US23 / 34768 and its progeny, US Patent App. No. 18 / 879,809 and its progeny, and / or US Patent App. No. 18 / 945,391 and its progeny, the entireties of which are hereby incorporated by reference into this disclosure. For example,haptic device 1002 may be similar to nodes 104, 106, and 108 but with the differences now described, including a housing 1004 configured to couple with an attachment element 1006 and related differences and benefits associated therewith.

[0108] Housing 1004 may have a small footprint (e.g., with a circular shape having a diameter that is approximately equal to that of a US quarter) that, in combination with attachment element 1006, allows haptic device 1002 to be quickly and easily put on or taken off by the user. The small footprint of housing 1004 also allows multiple separate haptic devices 1002 to be worn by the user. Haptic device 1002 which may be worn similarly to a watch. In one example, attachment element 1006 is a watch band. In other examples, attachment element 1006 may be a necklace, sleeve, clothing, etc. Multiple haptic devices 1002 may be coupled with attachment element 1006 such that a user may put on and take off multiple haptic devices 1002 simultaneously.

[0109] Housing 1004 may have any desired shape. Only one haptic device 1002 is housed by housing 1004 in FIGs. 10-14. However, housing 1004 may be sized and shaped to receive multiple haptic devices 1002 therein. Housing 1004 may be made with additive manufacturing. Housing 1004 may be 3D printed. Housing 1004 may be transparent or translucent.

[0110] FIG. 11 depicts haptic device 1002 without attachment element 1006. Housing 1004 may include first and second arms 1008a and 1008b coupled with a pin 1010. Pin 1010 may be detachably coupled to first and second arms 1008a and 1008b. Pin 1010 may be positioned within an opening of attachment element 1006 and then coupled with first and second arms 1008a and 1008b so as to couple attachment element 1006 with housing 1004. In some examples, pin 1010 is a watch pin, a spring bar, and / or like structures, including those commonly used with an Apple® Watch and related technologies.

[0111] Haptic device 1002 may be configured to transfer power or data between an outside source and haptic device 1002. Housing 1004 may include a port 1012 that allows a transfer of power or data. For example, port 1012 may be a Cat3, Cat5E, Cat6, USB- A, USB-B, USB-C, HDMI, etc. Alternatively, data may be transmitted between haptic device 1002 and an outside source via one or more wireless communication protocols. The wireless communication protocol may be Bluetooth, Zigbee, Wi-Fi, near field communication, etc. Haptic device 1002 may be in two-way communication with an outside device or controller via the wirelesscommunication protocol. The controller may be similar to controller 110 but include wireless communication capabilities.

[0112] Still referring to FIG. 11, a cover 1014 may be coupled to housing 1004. Cover 1014 may be configured to transfer heat between one or more components of haptic device 1002 and the environment via natural convention. FIG. 12 shows a top perspective view of haptic device 1002 with cover 1014 removed. Haptic device 1002 may include a vibration element. The vibration element may emit a vibration energy. Haptic device 1002 may include first and second vibration elements 1016a and 1016b. Vibration elements 1016a and 1016b may be a piezoelectric actuator, linear resonant actuator, etc. In some examples, vibration elements 1016a and 1016b are the same. In other examples, vibration elements 1016a and 1016b are different. Vibration elements 1016a and 1016b may be described as electromechanical actuators that are independently controlled. For example, the vibration frequency or amplitude of one of vibration elements 1016a and 1016b may be different from that of the other of vibration elements 1016a and 1016b. Vibration elements 1016a and 1016b may be spaced from each other by a spacing distance that is greater than a diameter of each of the vibration elements 1016a and 1016b. This spacing distance may help ensure that a user’s skin is able to detect the presence of two vibration elements rather than a single source.

[0113] Haptic device 1002 may include one or more spring-loaded pins 1018 (as known as pogo pins) operable as an electrical connector mechanism with spring plungers that are used for improved durability over other electrical contacts, and to increase the resilience of their electrical connection to mechanical shock and vibration. Pins 1018 may help the components of haptic device 1002 to vibrate in unison or close to in unison. This may help reduce relative vibration between the components of haptic device 1002. As shown in FIG. 13, pins 1018 may be utilized to establish a durable and resilient electrical connection between thermal sensor 1028 and substrate 1020 described below.

[0114] Haptic device 1002 may include a substrate 1020 configured to support one or more components within housing 1004. Substrate 1020 may be positioned within a recess defined by housing 1004. Substrate 1020 may be a printed circuit board that establishes an electrical communication with one or more components of haptic device 1002 and mechanically supports those elements inside of housing 1004. For example, elements such as port 1012, vibration elements 1016a, 1016b, plurality of light emitters 1032, and TEC module 1024 may beelectrically coupled with and / or structurally supported by substrate 1020, making it a central component of haptic device 1002.

[0115] FIG. 13 depicts an exploded view of haptic device 1002. Substrate 1020 may include a first side and a second side opposite the first side. An opening 1022 may extend through substrate 1020 from the first side to the second side. Opening 1022 may be a central opening. Opening 1022 may be centrally located on substrate 1020.

[0116] A plurality of light emitters 1032 may be coupled to substrate 1020. Light emitters 1032 may be light emitting diodes. Light emitters 1032 may be coupled to the second side of substrate 1020 such that light is directed toward a skin facing side of haptic device 1002 in directions transverse with the skin and / or away from a perimeter edge of haptic device 1002 in directions parallel to the skin. Light emitters 1032 may be spaced around a perimeter of substrate 1020. Light emitters 1032 may surround opening 1022. Light emitters 1032 may each be the same. Light emitters 1032 may be independently controlled by the controller. Alternatively, light emitters 1032 may be controlled as one or more groups. Haptic device 1002 may include a power source that is electrically coupled with, and provides power to, light emitters 1032. Light emitters 1032 may emit a plurality of wavelengths. Light emitters 1032 may be configured to administer red light therapy to a user. All or portions of housing 1004 may be transparent or translucent such that haptic device 1002 may be configured to direct light toward the skin and / or illuminate adjacent portions of the skin when light emitters 1032 emit light energy, making it possible to affect the user’s skin and underlying tissues and / or communicate with the user’s brain via their eyes.

[0117] A TEC (thermoelectric cooler) module 1024 may be positioned within opening 1022. In some examples, when haptic device 1002 is assembled, a first portion of TEC module 1024 may be positioned on the first side of substrate 1020, a second portion of TEC module 1024 may be positioned on the second side of substrate 1020, and perimeter edges of TEC module 1024 may be spaced apart from substrate 1020 to limit transfers of heat and / or vibration therebetween. The power source may be coupled with TEC module 1024 such that TEC module 1024 absorbs or emits heat when an electrical current passes through TEC module 1024. For example, TEC module 1024 may be operable to realize a Peltier effect, causing it to absorb heat when current passes in a first direction; and / or a Seebeck effect, causing it emit heat when current passes in a second direction.

[0118] TEC module 1024 may be coupled with a user interface 1026 configured to contact a user’s body (e.g., skin or hair). In other examples, user interface 1026 may be placed sufficiently close to user’s body without contact user’s body but still allowing for transfer of energies described herein. User interface 1026 may be thermally conductive such that heat may be transferred between TEC module 1024 and the user’s skin. Haptic device 1002 may include a thermal sensor 1028. Thermal sensor 1028 may be thermally coupled with user interface 1026. Thermal sensor 1028 may detect a temperature of the user’s skin. The controller may measure the rate of change in temperature of the user’s skin over time. Thermal sensor 1028 may abut a first side of user interface 1026 and a second side of user interface may be positioned in contact with or adjacent to the user’s skin.

[0119] Haptic device 1002 may include a second cover 1030 configured to couple with housing 1004. Second cover 1030 may be positioned adjacent to or in contact with the user’s skin. In some examples, second cover 1030 is coupled to housing 1004 via a fastener. In other examples, second cover 1030 may be coupled to housing via adhesive or ultrasonic weld. In still other examples, housing 1004 and second cover 1030 are a monolithic element, such as when they are 3D-printed from a transparent or translucent such material configured to distribute light output from emitters 1032. As shown in FIG. 13, second cover 1030 may be transparent or translucent such that light from light emitters 1032 may be directed via placement of emitters 1032 to pass through second cover 1030 in directions that are transverse and / or parallel with the user’s skin.

[0120] Second cover 1030 may include a central opening 1034 configured to receive a portion of user interface 1026. The portion of user interface 1026 may extend through central opening 1034 such that user interface 1026 is placed in contact with a user’s body. In keeping with FIG. 13, second cover 1030 may define an annular lens with a concave cross-section configured to spread light output from emitters 1032, such as when using the light to communicate with the brain; or a convex cross-section configured to focus light output from emitters 1032, such as when using the light to apply an intensive therapy, such as red-light therapy. User interface 1026 may be thermally coupled to a skin-facing side of TEC module 1024 and manufactured from a highly thermally conductive medium (e.g., aluminum, gold, or other highly thermally conductive medium with equally high degrees of biocompatibility and durability, such as lab-manufactured diamond or like crystals). User interface 1026 may be relatively thin such that even small amounts of heat may pass therethrough with minimal lossesso as to improve operational efficiencies and minimize heat gains. For example, user interface 1026 and second cover 1030 may each be 3D printed from a thin layer of thermoplastic or thermally conductive silicone, or otherwise formed from an alternative material.

[0121] As shown in FIG. 13, an O-ring 1036 may be positioned between user interface 1026 and second cover 1030. O-ring may create a watertight seal between user interface 1026 and second cover 1030. O-ring may dampen any relative vibration between second cover 1030 and user interface 1026 when vibration elements 1016a and 1016b emit vibration energy.

[0122] Referring to FIG. 15, cover 1014 may be a heat sink that transfers heat from haptic device 1002 to the surrounding environment. Cover 1014 may include a thermal pad 1038 configured to thermally couple to TEC module 1024. Cover 1014 may include a first side and a second side opposite the first side. Thermal pad 1038 may be positioned on the second side. The first side may include one or more features to increase heat transfer between cover 1014 and the environment. For example, cover 1014 may include one or more fins on the first surface. Cover 1014 may be manufactured from a thermally conductive material (e.g., milled from aluminum). Cover 1014 may include one or more recesses 1040 configured to receive at least a portion of one of vibration elements 1016a and 1016b, each of which may be fixedly attached to cover 1014 and thus configured to transfer vibrational energies directly thereto. A majority of the first vibration element 1016a may be positioned in the recess 1040. Positioning the vibration element within the recess may provide a more compact haptic device 1002, and further promote the transfer of vibrational and / or thermal energies directly to cover 1014.

[0123] Fig. 16 depicts another embodiment of a haptic device 1602. Haptic device 1602 may be the same as haptic device 1002 include a housing 1604 that is different from housing 1004. At least a portion of housing 1604 may be received by a receiver 1606. Housing 1604 may include a first coupling element 1608 and receiver 1606 may include a second coupling element 1610. One of first and second coupling elements 1608 and 1610 may be a protrusion and the other of first and second coupling elements 1608 and 1610 may be a recess configured to receive the protrusion. First and second coupling elements may be threadedly engaged so as to couple housing 1604 to receiver 1606. In some examples, receiver 1606 functions similarly to second cover 1030. Housing 1604 and receiver 1606 may each be transparent or translucent such that haptic device 1602 glows when light emitters emit light.

[0124] FIG. 17 depicts an example of a user wearing two haptic devices 1702 and 1704, each of which may be like haptic devices 102 and / or 1002 described above. Haptic devices 1702 and 1704 may receive different haptic signals and be operable to output coordinated bilateral stimulus to the user, such as one that that communicates frequency differences using different types of energy, such as vibration and light. The haptic devices 1702 and 1704 may receive different haptic signals which may cause them to output energies at different frequencies. The user’s brain may process the haptic energy outputs as a difference of the frequencies, like a binaural beat, but with a more wholistic approach that goes beyond mere audio, making them equally accessible to both sensory-able users and sensor-impaired users. For example, a first haptic signal may cause first haptic device 1702 to output vibrational energies toward the user’s skin at a first frequency of 300 Hertz and a second haptic signal may cause second haptic device 1704 to output vibrational energies toward the user’s skin at a second frequency of a second frequency of 250 Hertz so that user’s brain gradually falls into synchrony with the difference, causing the user to perceive a synchronous frequency of a 50 Hz that is commonly associated with high levels of focus, making it possible for an attending therapist to utilize the communication system described herein administer this type of energy prescription to patients struggling with focus, such as those with ADHD, or memory loss, such as those with Alzheimer’s disease. This bilateral stimulation may be used with other types of output energies such as sound, light, heat, etc. The haptic signals may be received from the haptic track composition and energy prescription system illustrated in FIGS.3A-3B

[0125] Haptic devices 1702 and 1704 may be multi-energy haptic devices. Haptic devices 1702 and 1704 may be haptic device 102, haptic device 334, haptic device 404, knee brace 502, nodes 602 and 604, haptic device 1002, or haptic device 1602. In some examples, first and second haptic devices 1702 and 1704 may output energies at any desired frequency selected independently from each other. In other examples, the output frequency of the second haptic device 1704 may be determined based on the output frequency of the first haptic device 1702. The delta between the output frequencies may be selected to produce a desired effect on the user even as the output frequency of the first haptic device changes. For example, the output frequency of the second haptic device 1704 may be 4.99 Hz greater than the output frequency of the first haptic device 1702 so as to induce heart relaxation.

[0126] Haptic devices 1702 and 1704 may be haptic generators comprising generator elements adapted to output a plurality of different haptic energies. For example, each generatorelement may comprise a separate, standalone electromechanical device that is independently operable to output one type of haptic energy toward the user’ s skin, making it possible to output different combinations of a plurality of different haptic energy types toward the skin.

[0127] Haptic devices 1702 and 1704 may be positioned on different sides of the user’s body. For example, haptic devices 1702 and 1704 may be positioned on a user’s left and right wrist. Haptic devices 1702 and 1704 may be integrated into a wearable device configured for bilateral haptic energy delivery. The bilateral stimulus may also be applied to different parts of the body. For example, haptic devices 1702 and 1704 may be coupled to an arm and leg, respectively. Bilateral stimulation may allow the user to perceive a frequency that is outside the range of frequencies provided by haptic devices 1702 and 1704.

[0128] A haptic controller may send signals to activate haptic devices 1702 and 1704. The haptic controller may be node controller 322 or may be similar to controller 110 but include wireless communication capabilities. The haptic controller may be operatively connected to a haptic instrument such as haptic instrument 1812 illustrated in FIG. 19. The haptic controller may send a first haptic signal to haptic device 1702 and a second haptic signal to haptic device 1704. The haptic controller may be operable to cause first and second haptic generators to output a coordinated bilateral stimulus to first and second sides of a user’s body. The coordinated bilateral stimulus may communicate a frequency difference between the different haptic energies.

[0129] The haptic controller may adjust at least one of the first and second haptic signals to maintain a constant frequency difference between the first and second haptic generators. The haptic controller may include a timer configured to adjust the haptic energy output of haptic devices 1702 and 1704 according a time-based progression to guide a user through stages of relaxation, focus, or sleep. The haptic controller may be adapted to analyze environmental frequency noise using an integrated sensor and adjust the first and second haptic signals to optimize the communicated frequency difference. At least at least one of the first and second haptic signals may be adjusted to achieve a desired mental or physiological outcome. For example, the haptic controller may include a biometric sensor that senses an environmental or physiological condition; and the first and / or second haptic signals may be adjusted in response to a change in a physiological parameter sensed by the biometric sensor.

[0130] FIG. 18 depicts an example of a group haptic experience implemented via a haptic system. Aspects of the haptic devices and haptic systems described herein may allow for the simultaneous control of a plurality of haptic devices on or adjacent a plurality of users. Haptic device users may be part of an audience in an arena, stadium, or other area or venue where large or small groups may convene. A haptic system may include a haptic instrument and one or more haptic devices. The haptic system may also include a communication network that allows communication between the haptic instrument and the one or more haptic devices. A composer may send one or more haptic signals from a haptic instrument to the haptic device users via one or more wireless communication protocols over the communication network. The haptic devices may include a computer chip such as a Qualcomm® Snapdragon® chip that allows wireless communication with the haptic instrument or other haptic devices and / or another type of system-on-chip (or “SoC”) semiconductor, like those that are currently available and operable to power mobile devices in a fast, secure, and power-efficient manner.

[0131] In one example, the arena may be equipped with an arena network such as a highspeed Wi-Fi network or mesh network. The arena network may broadcast the haptic signal to any haptic devices connected to the arena network. In other examples, one or more of the haptic devices may function as a node in a mesh network. The haptic signal is sent from the composer to one or more haptic devices which then repeat the haptic signal, via Zigbee communication protocol, thereby daisy chaining the haptic signal to all the haptic devices throughout the arena. The wireless communication protocol may be message queuing telemetry transport (MQTT), Bluetooth, Zigbee, Z-Wave, or near field communication (NFC). The haptic instrument may send a haptic signal to a server having memory and a processor, and the server may send the haptic signal over the arena network to the haptic devices. The haptic instrument may be a user input 724 associated with the haptic track composition and energy prescription system illustrated in FIG. 7. The haptic track may be distributed to the haptic devices in accordance with the haptic track composition and energy prescription system illustrated in FIG. 7.

[0132] As shown in FIG. 18, a composer 1802 may perform for an audience 1804 including one or more members 1806. Composer 1802 may conduct a music performance, a haptic performance, or a combination thereof. Composer 1802 may send haptic signals from a haptic instrument 1812 to a haptic device 1808 on or adjacent to at least some members 1806 of the audience. Composer 1802 may conduct a music performance while sending haptic signals to haptic devices 1808.

[0133] Haptic device 1808 may be configured to output different patterns of energy outputs, including any combination of audio, haptic, aroma sensory, and / or visual energy outputs, each of which may be simultaneously output individually and / or in combination toward the user in a precise, repeatable way during the performance responsive to the haptic signal. Haptic device 1808 may be haptic device 102, haptic device 334, haptic device 404, knee brace 502, nodes 602 and 604, haptic device 1002, haptic device 1702, a smartwatch, smartphone, or other DataFeel technology. Haptic device 1808 may include generating elements that are independently operable to output a plurality of different energy types. At least some of those energy types may be haptic energies. Haptic device 1808 may also be referred to as a generator.

[0134] In some examples, audience members 1806 wear haptic device 1808. In other examples, audience members 1806 grip or hold haptic device 1808. In still other examples, haptic device 1808 may be a fixture in an arena where the audience 1804 is located. For example, one or more haptic devices 1808 may be fixed to a seat and audience member 1806 may sit in the seat during a performance. If haptic device 1808 is movable, haptic devices 1808 may be distributed to audience members 1806 as part of a ticket sale. Haptic devices 1808 may be distributed to members 1806 upon arrival at the venue. Audience members 1806 may bring their own haptic devices 1806 to the arena.

[0135] Composer 1802 may utilize an observable output device 1810 during the performance to create audio output, visual output, or a combination thereof. Observable output device 1810 may be a mixing board, turntable, projector, or a musical instrument such as a guitar, bass, saxophone, keyboard, keytar, etc. Some observable output devices 1810 allow composer 1802 to conduct a live performance. Other observable output devices 1810 allow reproduction of previously recorded or streaming content.

[0136] Composer 1802 may utilize haptic instrument 1812 during a performance to output haptic signals to haptic devices 1808. Haptic instrument 1812 may be a composition tool that generates and transmits haptic signals to haptic devices 1808. Haptic instrument 1812 may communicate with haptic devices 1808 via any known wired and / or wireless networking technology. Haptic instrument 1812 may simultaneously control a plurality of haptic devices 1808. Haptic instrument 1812 may simultaneously control all of the haptic devices 1808 of the audience members 1806. Therefore, composer 1802 may simultaneously perform an audioperformance as well as a haptic performance utilizing haptic instrument 1812 and observable output device 1810. Although only one composer 1802 is shown in FIG. 18, it is understood that there may be any number of composers. For example, some composers may play musical instruments as part of a musical group while another composer performs a haptic track via haptic instrument 1812.

[0137] Composer 1802 may control the haptics in real time by manipulating haptic instrument 1812, as discussed in greater detail below. Composer 1802 may control the haptics in a similar fashion to a DJ adjusting an equalizer (“EQ”) or volume. In some examples, composer 1802 physically interacts with haptic instrument 1812 to create the haptic signals. In other examples, haptic instrument 1812 transforms audio data from the music to create the haptic signals via haptic map 306 and other components of the haptic track composition and energy prescription system illustrated in FIGS. 3A-3B.

[0138] A display 1816 may provide ongoing visualization of the physiological data and the haptics synced with the music in real time. Display 1816 may include a television, mobile computing device, or other user device with a display that may be adapted to GUI 202 or GUI 402. Haptic devices 1818 may be communicable with display 1816 via any known wired and / or wireless networking technology. Haptic instrument 1812 may monitor the status of each haptic device 1808. Haptic instrument 1812 may send the status information for presentation on display 1816.

[0139] The system described herein may pixelate the audience 1804, with each haptic device 1806 acting as a pixel, making it possible to create group experiences that transcend place and time. Pixelation may allow the composer 1802 to illuminate select individual haptic devices, for example, so as to create a desired visual imagery within the arena. The location of each haptic device 1808 may be transmitted to and stored in memory 714 of the server computing device 715 of the haptic track composition and energy prescription system illustrated in FIG. 7. In some examples, the system may detect a location of each haptic device 1808. In other examples, each haptic device may report its location to the server computing device 715. The relative positions of the haptic devices 1808 may be shown on display 1816 to create a virtual canvas and the composer may create a desired imagery on the canvas. Pixelation may also allow the composer to induce movement of select members of the crowd by activatinga haptic device 1808. For example, a haptic signal may be sent to haptic device 1808 on select audience members’ seats to induce the audience members to stand up.

[0140] Other movements of the audience members may also be induced. For example, audience members 1806 may wear or hold more than one haptic device 1808. The haptic devices 1808 may be coupled to different locations of member’s body. Composer 1802 may encourage simultaneous movement by the members 1806 of the audience 1804 by simultaneously activating the haptic devices coupled to the same limb (e.g., right leg) of each member 1806. Composer 1802 may wear a composer device 1814 that allows composer 1802 to feel the physiological data in real time via an audience-specific haptic signal comprising combinations of light, temperature differential, or vibration. By way of example, each haptic device worn by each audience member may comprise one or more physiological sensors operable to output physiological data associated, and the haptic controller may be configured to collect the data, aggregate it into the audience-specific signal, and output a haptic representation of the signal back to composer 1802, allowing them to react to the signal in real time without looking at a screen or otherwise diverting their attention from the audience members.

[0141] FIG. 19 illustrates haptic instrument 1812 that generates signals which may comprise a haptic track. Haptic instrument 1812 may be adapted to receive user input representing a desired energy output. Haptic instrument 1812 may be a musical instrument that may be played to generate haptic signals. Haptic instrument 1812 may output musical notes and haptic signals simultaneously in real time. Haptic instrument 1812 may include a variety of inputs, with different subsets of the inputs mapped to different haptic devices, such that the haptic instrument 1812 may output haptic signals to each of the haptic devices. For example, haptic instrument 1812 may be a keyboard including a plurality of octaves defined by keys. A first octave 1818 of keys may be mapped to one or more first generating elements of a first haptic device, like first members of an orchestral group. A second octave 1820 of keys may be mapped to one or more second generating elements of a second haptic device, like second members of the orchestral group. In some examples, different subsets of an octave are mapped to different haptic output devices and / or elements thereof; and the keys within each octave may be mapped to different haptic energy output frequencies associated therewith.

[0142] A composer may output haptic signals from haptic instrument 1812 to the first haptic device by playing notes within the first octave 1818. The composer may output haptic signals from haptic instrument 1812 to the second haptic device by playing notes within the second octave 1820. The composer may output haptic signals from haptic instrument 1812 to each of the first and second haptic devices by playing notes within the first and second octaves 1818 and 1820.

[0143] Haptic instrument 1812 may switch between haptic energy output types of the haptic devices. A composer may toggle between haptic energy output types by toggling a switch, inputting a selected keystroke sequence, or having certain keys mapped to different energy output types on haptic instrument 1812. In some examples, the toggle is a switch, knob, button, foot pedal, etc. Haptic instrument 1812 may include a pitchbend wheel 1822 and / or a modulation wheel 1824 that allows selection of different haptic energy output types. Haptic instrument 1812 may output a signal causing the haptic device to output a first haptic energy output type when the modulation wheel 1824 or pitchbend wheel 1822 is in a first position and output a second haptic energy output type when the modulation wheel 1824 or pitchbend wheel 1822 is in a second position. The first position may be different than the second position. The composer may move the modulation wheel 1824 or pitchbend wheel 1822 between first and second positions to toggle between the first and second haptic energy output type.

[0144] Haptic instrument 1812 may switch between haptic energy output types in response to a selected keystroke sequence. For example, composer may play C#, D#, then C# to switch from a first haptic energy output type to a second haptic energy output type.

[0145] Haptic instrument 1812 may have inputs that allow a user to select the haptic energy output type without toggling a switch. For example, a single haptic device may be mapped to more first and second octaves 1818 and 1820. First octave 1818 may be mapped to a first energy output type and second octave 1820 may be mapped to a second energy output type. A composer may select the haptic energy output type by playing notes within the fist and second octaves 1818 and 1820 as desired.

[0146] The composer may compose or create a haptic track by playing notes on haptic instrument 1812. It may be desirable to have pre-recorded haptic outputs stored in the haptic track composition system as described in reference to FIGs 3 A-3B. A mini haptic energy output track may be mapped to a combination of keystrokes on haptic instrument 1812. For example,the composer may play C#, C#, C# within first or second octave 1818 or 1820 to activate a haptic track sequence for the haptic device mapped to that octave. The haptic track may be operable to cause first and second haptic generators to output a coordinated stimulus by causing first haptic generating elements to output a first plurality of different haptic energies and second generating elements to output a second plurality of different energy types.

[0147] The haptic track may include information regarding the amplitude of haptic energy output from the haptic device. The composer may control the amplitude of haptic energy output from the haptic device based on keystroke velocity while playing haptic instrument 1812. For example, a greater keystroke velocity may correspond to a higher haptic output amplitude. In other examples, the amplitude of haptic energy output may be controlled by modulation wheel 1824. The modulation wheel 1824 may be moved to a desired setting similar to adjusting a volume knob.

[0148] The haptic signals output by haptic instrument 1812 may be recorded as a haptic track. The haptic track may be stored in haptic track library 816 as illustrated in FIG. 8. The haptic track may be output in MIDI, MP3, DMX512, or other known formats. The system described herein may include an interface that automatically converts musical notes into a haptic track. The interface may comprise haptic map 306 as illustrated in FIG. 3A to convert musical notes to haptic signals. The composer may create the haptic track in real time, while playing along with a musical track. The haptic track may be synced to the audio track to provide a listener with an immersive audio and haptic experience. The composer may record a first haptic track for a first haptic device and a second haptic track for a second haptic device. The first and second haptic tracks may be combined into a single haptic track.

[0149] Haptic tracks may be composed or created to accompany a music track and produce desired effects while listening to the music track, regardless of where the user is located. Haptic devices may be mapped to select elements of the music track. A first haptic device may be mapped to the percussion element of the music track. A second haptic device may be mapped to the melody of the music track. A third haptic device may be mapped to the vocals of the music track.

[0150] FIG. 20 depicts a composer 2002 performing with a haptic system in front of an audience including a first audience section 2004A, a second audience section 2004B, and a third audience section 2004C. Composer 2002 may be playing a haptic instrument 2006 whileoperating an activator 2008. Haptic instrument 2006 may be haptic instrument 1812 or another type of haptic track output device. Activator 2008 may be operable to selectively activate or deactivate haptic devices within one or more of first, second, and third audience sections 2004A, 2004B, and 2004C.

[0151] At least some of the audience members may wear, or be adjacent to, haptic devices 2010. Haptic devices 2010 may be haptic device 102, haptic device 334, haptic device 404, knee brace 502, nodes 602 and 604, haptic device 1002, haptic device 1702, haptic device 1808, a smartwatch, or a smartphone. As explained above with respect to haptic devices 1808, haptic devices 2010 may be connected to haptic instrument 2006 via wireless or wired connection. Similarly, activator 2008 may be electrically coupled to haptic devices 2010 via wireless connection. Activator 2008 may emit a signal that is received by haptic devices 2010 to activate or deactivate haptic devices 2010 within selected sections of the first, second, and third audience sections 2004 A, 2004B, and 2004C. Activator 2008 may simultaneously activate or deactivate all of the haptic devices 2010 within the selected one of the first, second, and third audience sections 2004 A, 2004B, and 2004C. Activator 2008 may be a head mounted device such as smart glasses having onboard circuitry for indicating the operator’s filed of view when wearing the head mounted device. Activator 2008 may be a wand having onboard circuitry for indicating what the activator is pointing at. Activator 2008 be camera-readable indicia for indicating what activator 2008 is pointing at. For example, the camera-readable indicia may be on coupled to or disposed on composer such that composer 2002 may activate haptic devices 2010 by performing a gesture.

[0152] A sensor may detect a position of activator 2008. relative to the first, second, and third audience sections 2004A, 2004B, and 2004C. The sensor may be a camera. The sensor may be configured to detect a gesture of the composer 2002 to activate or deactivate haptic devices 2010. Composer 2002 may activate haptic devices within a select audience section by pointing or otherwise aligning activator 2008 with the select audience section. Composer 2002 may activate select haptic devices by moving activator 2008 in a pattern recognizable by the sensor. Activator 2008 may be a separate device from haptic instrument 2006. Composer 2002 may emit haptic signals via haptic instrument 2006 while controlling activation of haptic devices 2010 with activator 2008. A first composer may operate activator while a second composer operates haptic instrument.

[0153] Aspects of this disclosure may be implemented in digital circuits, computer- readable storage media, as one or more computer programs, or a combination of one or more of the foregoing. The computer-readable storage media may be non-transitory, e.g., as one or more instructions executable by a cloud computing platform and stored on a tangible storage device.

[0154] The phrase “configured to” is used in different contexts related to computer systems, hardware, or part of a computer program. When a system is said to be configured to perform one or more operations, this means that the system has appropriate software, firmware, and / or hardware installed on the system that, when in operation, causes the system to perform the one or more operations. When some hardware is said to be configured to perform one or more operations, this means that the hardware includes one or more circuits that, when in operation, receive input and generate output according to the input and corresponding to the one or more operations. When a computer program is said to be configured to perform one or more operations, this means that the computer program includes one or more program instructions, that when executed by one or more computers, causes the one or more computers to perform the one or more operations.

[0155] Unless otherwise stated, the foregoing alternative examples are not mutually exclusive, but may be implemented in various combinations to achieve unique advantages. As these and other variations and combinations of the features discussed above may be utilized without departing from the subject matter defined by the claims, the foregoing description of the embodiments should be taken by way of illustration rather than by way of limitation of the subject matter defined by the claims. In addition, the provision of the examples described herein, as well as clauses phrased as “such as” “including” and the like, should not be interpreted as limiting the subject matter of the claims to the specific examples; rather, the examples are intended to illustrate only one of many possible embodiments. Further, the same reference numbers in different drawings may identify the same or similar elements.

Claims

CLAIMS1. A system comprising: a first energy generator comprising first generating elements that are independently operable to output a first plurality of different energy types; a second energy generator comprising second generating elements that are independently operable to output a second plurality of different energy types; a musical instrument configured to receive user input representing an energy output and generate input signals in response to receiving the user input, the musical instrument including a plurality of data inputs defining a first subset and a second subset, the first subset being mapped to the first generating elements of the first generator and the second subset mapped to the second generating elements of the second generator; and a control unit operably connected to the musical instrument, the control unit configured to analyze the input signals and generate an energy track by encoding the input signals into a musical instrument digital interface format, the energy track being operable to cause the first energy generator and the second energy generator to output a coordinated stimulus by causing the first generating elements to output one or more energy types of the first plurality of different energy types and the second generating elements to output one or more energy types of the second plurality of different energy types.

2. The system of claim 1, wherein the instrument comprises an electric piano keyboard and the plurality of data inputs comprise keys on the electric piano keyboard.

3. The system of claim 2, wherein the first and second subsets comprise a first and second octave of keys, respectively, on the electric piano keyboard.

4. The system of claim 2, wherein: the electric piano keyboard includes a modulation wheel moveable from a first position to a second position; andthe first position and the second position are mapped to one energy type of the first and second pluralities of different energy types.

5. The system of claim 2, wherein a velocity of a keystroke is mapped to one energy type of the first and second pluralities of different energy types.

6. The system of claim 1, further comprising a storage medium that is configured to store the energy track for playback with an audio track.

7. The system of claim 1, wherein the energy track is generated in real time as the user input is received.

8. The system of claim 1, wherein the musical instrument outputs musical notes and generates the input signals simultaneously.

9. The system of claim 8, wherein the control unit is configured to map the musical notes to energy signals.

10. The system of claim 1, wherein the control unit comprises an interface enabling a user to modify the generated energy track.

11. The system of claim 1, wherein the machine-readable format comprises a time- sequenced series of instructions interpretable by the first and second energy generators.

12. The system of claim 1, wherein the first and second pluralities of different energy types comprise at least two of an electrical energy, an optical energy, a pressure energy, a thermal energy, and a vibratory energy.

13. A method compri sing : capturing user input with a musical instrument having a plurality of data inputs defining a first subset and a second subset, the first subset mapped to a first energy generator and the second subset mapped to a second energy generator; processing the captured user input to extract event data;converting the event data into a musical instrument digital interface format; and outputting the musical instrument digital interface format as an energy track operable to activate the first and second energy generators.

14. The method of claim 13, wherein the musical instrument comprises an electric piano keyboard, the plurality of data inputs comprise keys on the electric piano keyboard, and capturing user input includes capturing a keystroke by the user.

15. The method of claim 14, wherein the plurality of data inputs includes a plurality of keys defining the first subset of data inputs and a second plurality of keys defining the second subset of data inputs.

16. The method of claim 13, comprising synchronizing the energy track with an audio track.

17. The method of claim 13, comprising outputting the energy track to the first and second energy generators thereby causing the first and second haptic generators to output an energy signal responsive to the energy track.

18. The method of claim 17, wherein the event data includes timing, intensity, and frequency of the energy signal.

19. The method of claim 18, wherein the energy signal comprises at least two of an electrical energy, an optical energy, a pressure energy, a thermal energy, and a vibratory energy.

20. The method of claim 18, comprising receiving an additional user input with the musical instrument and modifying the energy signal responsive to the additional user input.

21. A system comprising:a first energy generator comprising first generator elements configured to output a first plurality of different energy types on a first side of a user’s body; a second energy generator comprising second generator elements configured to output a second plurality of different energy types on a second side of a user’s body; and a controller that is operable to cause the first energy generator and the second energy generator to output a coordinated bilateral stimulus to the first and second sides of the user’s body by generating and transmitting a first control signal and a second control signal to the first and second energy generators, respectively, so that the coordinated bilateral stimulus communicates a frequency difference between one or more energy types of the first plurality of different energy types and one or more energy types of the second plurality of different energy types.

22. The system of claim 21, wherein: the first energy generator outputs a first energy type of the first plurality of different energy types at a first frequency; the second energy generator outputs a second energy type of the second plurality of different energy types at a second frequency; and the second frequency is different from the first frequency to cause a user to perceive a third frequency at the difference between the first and second frequency.

23. The system of claim 21, wherein the second frequency changes by a second frequency delta in response to a change in the first frequency by a first frequency delta.

24. The system of claim 23, wherein the second frequency delta is equal to the first frequency delta.

25. The system of claim 21, wherein the frequency difference is about 4.99 Hz.

26. The system of claim 21 , wherein the first energy generator is coupled to a user’ s left wrist and the second energy generator is coupled to the user’s right wrist.

27. The system of claim 26, wherein the first and second energy generators are integrated into a wearable device configured for bilateral frequency delivery.

28. The system of claim 21, wherein the controller includes a timer configured to operate the first and second haptic generators according to a time-based progression to guide a user through stages of relaxation, focus, or sleep.

29. The system of claim 21, wherein the controller is configured to analyze environmental frequency noise using an integrated sensor and adjust the first and second frequencies signals to optimize the frequency difference.

30. The system of claim 21, wherein the third frequency is adjusted by adjusting the first and second frequencies to achieve a desired mental or physiological outcome.

31. The system of claim 21, comprising a musical instrument comprising an electric piano keyboard configured to send the first and second energy signals to the first and second energy generators, respectively.

32. The system of claim 21, wherein the controller is configured to wirelessly communicate with the first and second energy generators.

33. The system of claim 21, wherein the first and second plurality of different energy types comprise at least two of an electrical energy, an optical energy, a pressure energy, a thermal energy, and a vibratory energy.

34. A method of generating bilateral stimulation, comprising: generating, by a controller, a first control signal comprising a first frequency; generating, by the controller, a second control signal comprising a second frequency; transmitting, by the controller, the first control signal to a first energy generator coupled to a first side of a user’s body, thereby causing the first energy generator to output a first combination of vibration and light at the first frequency; andtransmitting, by the controller, the second control signal to a second energy generator coupled to a second side of the user’s body, thereby causing the second energy generator to output a second combination vibration and light at the second frequency.

35. The method of claim 34, further comprising: adjusting, by the controller, the second control signal to maintain a constant frequency difference between the first and second energy generators.

36. The method of claim 34, wherein the first and second energy generators are further configured to output at least one of heat and electrical stimulation.

37. The method of claim 34, wherein the first energy generator is coupled to the user’s right wrist and the second haptic generator is coupled to the user’s left wrist.

38. The method of claim 34, further comprising: monitoring a physiological parameter of the user with a biometric sensor; and adjusting at least one of the first control signal and the second control signals in response to a change in the physiological parameter.

39. A haptic energy output system, comprising: a plurality of haptic generators on or adjacent a plurality of audience members; a haptic instrument operable by a performer, the haptic instrument operatively coupled with the plurality of haptic generators, the haptic instrument configured to receive user input representing a desired haptic output, generate haptic signals in real time in response to receiving the user input, and transmit the haptic signals to the plurality of haptic generators thereby causing the haptic generators to emit light and vibration.

40. The haptic energy output system of claim 39, wherein the haptic instrument is an electric piano keyboard.

41. The haptic energy output system of claim 40, wherein the electric piano keyboard includes a modulation wheel moveable from a first position to a second position, wherein the haptic signals cause the haptic generators to emit a first haptic energy type when the modulation wheel is in the first position, and wherein the haptic signals cause the haptic generators to emit a second haptic energy type when the modulation wheel is in the second position.

42. The haptic energy output system of claim 40, wherein the haptic instrument outputs musical notes and generates the haptic signals simultaneously.

43. The haptic energy output system of claim 39, further comprising: an activator configured to activate or deactivate at least one of the plurality of haptic generators.

44. The haptic energy output system of claim 43, wherein the plurality of haptic generators includes: a first plurality of haptic generators in a first section; a second plurality of haptic generators in a second section; and a third plurality of haptic generators in a third section, wherein the activator is configured to selectively activate or deactivate all of the haptic generators in one of the first, second, or third sections.

45. The haptic energy output system of claim 39, wherein the plurality of haptic generators includes a first plurality of haptic generators coupled to a right wrist of the audience members and a second plurality of haptic generators coupled to a left wrist of the audience members and the haptic instrument is operable to transmit a first haptic signal to the first plurality of haptic generators and a second haptic signal to the second plurality of haptic generators.

46. The haptic energy output system of claim 45, wherein the first and second haptic signals provide a coordinated bilateral stimulus that communicates frequency differences in vibration and light.

47. The haptic energy output system of claim 39, wherein the haptic instrument electronically communicates with the plurality of haptic generators via at least one of message queuing telemetry transport, Bluetooth, Zigbee, Z-Wave, or near field communication protocols.

48. The haptic energy output system of claim 39, wherein the haptic instrument is configured to sense a position of the haptic generators and transmit haptic signals to cause select ones of the plurality of haptic generators to emit light, thereby creating a selected visual effect across the audience.

49. The haptic energy output system of claim 39, wherein a first plurality of haptic generators are respectively coupled to seats in an arena and the haptic instrument is configured to induce movement of the audience members by causing the haptic generators coupled to the seats to vibrate.

50. The haptic energy output system of claim 39, wherein the haptic generators are configured to emit at least one of heat and electrical stimulation.

51. The haptic energy output system of claim 39, further comprising: a display configured to display a virtual representation of the haptic generators and relative positions within an arena, and a status of the haptic generators.

52. A method for generating haptic energy output, comprising: receiving, by a haptic instrument having a plurality of user inputs, a user selection indicating a desired haptic output; generating, by the haptic instrument, haptic signals based on the user input; transmitting, by the haptic instrument, the haptic signals to a plurality of haptic generators on or adjacent a plurality of audience members; and causing the plurality of haptic generators to emit light and vibration in response to the haptic signals.

53. The method of claim 52, wherein the haptic instrument comprises an electric piano keyboard including a modulation wheel, the method further comprising: transmitting a first haptic signal when the modulation wheel is in a first position, thereby causing the plurality of haptic generators to emit a first haptic energy type; and transmitting a second haptic signal when the modulation wheel is in the second position, thereby causing the plurality of haptic generators to emit a second haptic energy type.

54. The method of claim 53, further comprising simultaneously outputting musical notes and generating the haptic signals with the haptic instrument.

55. The method of claim 52, wherein the plurality of haptic generators are coupled to or adjacent to a plurality of users, and the method comprises: inducing movement of the users by causing the haptic generators to emit haptic energy.

56. The method of claim 52, further comprising: transmitting a first haptic signal to a first subset of the haptic generators; and transmitting a second haptic signal to a second subset of the haptic generators, wherein the first and second haptic signals a coordinated bilateral stimulus that communicates frequency differences in vibration and light of the first and second subsets of haptic generators.

57. The method of claim 52, wherein the haptic signals are transmitted to the plurality of haptic generators via at least one of Bluetooth, Zigbee, Z-Wave, or near field communication protocols.

58. The method of claim 52, wherein the generating step and the transmitting step are performed in real time in response to receiving the user selection.

59. The method of claim 52, further comprising activating or deactivating at least one of the plurality of haptic generators using an activator prior to transmitting the haptic signals.

60. The method of claim 59, wherein the plurality of haptic generators are divided into a first section, a second section, and a third section, and wherein the method includes selectively activating or deactivating all haptic generators in one of the sections using the activator.

61. The method of claim 59, further comprising simultaneously operating the haptic instrument and the activator to adjust the haptic output of at least one of the haptic generators.

62. The method of claim 59, wherein the activator comprises at least one of a head mounted device and a wand having onboard circuitry for indicating an orientation of the activator and the method comprises: determining that the activator is pointed at a subset of the haptic generators; and activating or deactivating the subset of haptic generators.

63. The method of claim 62, wherein the activator includes a camera-readable indicia for determining the orientation of the activator.

64. A method for generating haptic tracks and energy prescriptions, comprising: receiving, by one or more processors, input data; determining, by the one or more processors, control signals from the received input data; and causing, by the one or more processors, one or more energy generators to output energies responsive to the control signals.

65. The method of claim 64, further comprising receiving modifying data from a user.

66. The method of claim 65, further comprising causing the one or more energy generators to output modified energies responsive to the modifying data.

67. The method of claim 66, further comprising storing the modifying data to train a machine learning model.

68. The method of claim 64, wherein the input data includes MIDI data, MP3 data, temporal-spatial data related to musical notes, haptic layout information related to types of hardware, associated 3D layout and interconnection or data related to mapping of the musical notes to haptic signals.

69. The method of claim 64, wherein the control signals include pressure, heat, light, electrical stimulation, or vibration.

70. The method of claim 64, wherein the control signals are tailored for entertainment, therapy, education, security, communication, or medicine.

71. A system comprising: one or more processors; and one or more storage devices coupled to the one or more processors and storing instructions that, when executed by the one or more processors, cause the one or more processors to perform: receiving input data; determining control signals from the received input data; and causing one or more energy generators to output energies responsive to the control signals.

72. The system of claim 71, wherein the one or more processors are further caused to perform receiving modifying data from a user.

73. The system of claim 72, wherein the one or more processors are further caused to perform causing the one or more energy generators to output modified energies responsive to the modifying data.

74. The system of claim 73, wherein the one or more processors are further caused to perform storing the modifying data to train a machine learning model.

75. The system of claim 71, wherein the input data includes MIDI data, MP3 data, temporal-spatial data related to musical notes, haptic layout information related to types of hardware, associated 3D layout and interconnection, or data related to mapping of the musical notes to haptic signals.

76. The system of claim 71, wherein the control signals include pressure, heat, light, electrical stimulation, or vibration.

77. The system of claim 71, wherein the control signals are tailored for entertainment, therapy, education, security, communication, or medicine.

78. A non-transitory computer-readable medium for storing instructions that, when executed by one or more processors, cause the one or more processors to perform the method comprising: receiving input data; determining control signals from the received input data; and causing one or more energy generators to output energies responsive to the control signals.

79. The non-transitory computer-readable medium of claim 78, the method further comprising receiving modifying data from a user.

80. The non-transitory computer-readable medium of claim 79, the method further comprising causing the one or more energy generators to output modified energies responsive to the modifying data.

81. The non-transitory computer-readable medium of claim 80, the method further comprising storing the modifying data to train a machine learning model.

82. The non-transitory computer-readable medium of claim 78, wherein the input data includes MIDI data, MP3 data, temporal-spatial data related to musical notes, haptic layout information related to types of hardware, associated 3D layout and interconnection, or data related to mapping of the musical notes to haptic signals.

83. The non-transitory computer-readable medium of claim 78, wherein the control signals include pressure, heat, light, electrical stimulation or vibration.

Images