Systems and methods for immersive audio experiences

Abstract

A computer-implemented method for creating an immersive audio experience. The method includes receiving a selection of an audio track via a user interface and receiving audio track metadata for the audio track. The method includes querying an audio database based on the track metadata and determining that audio data for the audio track is not stored on the audio database. The method includes analyzing the audio track to determine one or more audio track characteristics. The method includes generating vibe data based on the one or more audio track characteristics, the vibe data including time-coded metadata. Based on the vibe data, generating visualization instructions for one or an A / V device in communication with a user computing device, and transmitting the generated visualization instructions and the audio track to the user computing device.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 63 / 178,983, filed April 23, 2021, the entirety of which is incorporated herein by reference. [Background technology]

[0002] The background discussion provided herein is for the purpose of generally presenting the context of the present disclosure. To the extent described in this Background section, the inventor's work, as well as aspects of the present specification that may not otherwise qualify as prior art at the time of filing, are not admitted expressly or impliedly as prior art to the present disclosure.

[0003] Traditionally, a music or audio listening experience at home may be a short-term in-person, live musical, or other audio experience. A live music experience may include many sensory inputs that are not traditionally available at home, or at least not available without significant investment in equipment and a great deal of work on behalf of the user. For example, a concert or other live music experience may include audio, visual, and other sensory inputs that may not traditionally exist in a home environment or may not be easily configured to provide an immersive music and audio experience. Summary of the Invention

[0004] The following presents a simplified summary of the disclosure in order to provide a basic understanding of some aspects of the disclosure. This summary is not an exhaustive overview of the disclosure. It is not intended to identify key or critical elements of the disclosure or to delineate the scope of the disclosure. The following summary is merely intended to present some concepts of the disclosure in a simplified form as a prelude to the more detailed description provided below.

[0005] In one embodiment, the present disclosure describes a system for creating immersive music and audio experiences in a user's home, vehicle, or other location. The experience may create an environment for a user to experience audio content, including music, podcasts, workouts, meditations, religious events, etc., in a sensory immersive way that allows the user to experience the music or other audio in a different way. This may include replicating, amplifying, or simulating a live music event, or may include non-live event artist motion graphics, or features designed through artificial intelligence that may differ from a live event, but may nevertheless be more immersive than a standard audio experience. In some embodiments, the system and method may include artist or other content creator "experience packs" that may include visual companions to their music, such as on-screen visualizers, bright colors, motion patterns, and stencils. The system may include various hardware components (e.g., LED lights, strobe lights, spotlights, lasers, speakers, projectors, etc.) that may be installed in the user's home, which may be connected to a cloud network, and software running on the user's computing device (e.g., desktop / laptop, cell phone, VR headset, tablet, etc.). In some embodiments, the result may be an immersive music or other audio experience that may include sound, lights, lasers, 3D projections, VR / AR experiences, and other visual or sensory elements that emulate attending a concert, other live music event, or an entirely AI-generated experience.

[0006] In some embodiments, systems and methods may use advanced technology and smart devices to transform harmonic intelligence from audio streams into stunning visual experiences. The process may use machine learning techniques to extract time-encoded metadata that is used to predict and design the ultimate experience, or "vibe," to accompany and complement the auditory content.

[0007] In some embodiments, this disclosure describes a computer-implemented method for creating an immersive audio experience. The method may include receiving a user selection of an audio track from a user computing device via a user interface and receiving audio track metadata for the audio track from an audio source. The method may include querying an audio database based on the track metadata. The method may include determining, based on the query, that audio data for the audio track is not stored on the audio database. The method may include analyzing the audio track to determine one or more audio track characteristics based on the determination that audio data for the audio track is not stored on the audio database and generating vibe data based on the one or more audio track characteristics. The vibe data may include time-coded metadata. Based on the vibe data, generate visualization instructions for one or more audio-visual (A / V) devices in communication with the user computing device. The method may include transmitting the generated visualization instructions and the audio track to the user computing device.

[0008] In another embodiment, the disclosure describes a system for creating an immersive audio experience. The system may include a plurality of audiovisual (A / V) devices configured to provide A / V effects. The system may include a user computing device configured to wirelessly communicate with the plurality of A / V devices. The user computing device may include one or more processors in communication with a memory including processor-executable instructions for providing a graphical user interface for receiving a user selection of an audio track. The system may include one or more remote computer servers in electronic communication with the user computing device. The one or more remote computer servers may include one or more processors in communication with a memory including processor-executable instructions for receiving audio track metadata for an audio track from an audio source and analyzing the audio track metadata to determine one or more audio track characteristics associated with the audio track. The memory may also include instructions for generating vibe data based on the one or more audio track characteristics, the vibe data may include time-coded metadata. The memory may also include instructions for generating visualization instructions for at least one of the plurality of audiovisual (A / V) devices based on the vibe data and transmitting the generated visualization instructions and the audio track to the user computing device. The memory of the user computing device may further include processor-executable instructions for transmitting the generated visualization instructions to at least one of a plurality of audiovisual (A / V) devices.

[0009] In another embodiment, the present disclosure describes a computer-implemented method for creating an immersive audio experience. The method may include receiving a user selection of an audio track from a user computing device via a user interface and analyzing the audio track to determine one or more audio track characteristics. The method may include generating vibe data based on the one or more audio track characteristics, the vibe data may include time-coded metadata. The method may also include receiving one or more user instructions from the user computing device to edit the vibe data to generate user vibe data. The method may include generating visualization instructions based on the user vibe data and transmitting the generated visualization instructions and the audio track to the user computing device. [Brief description of the drawings]

[0010] The present invention may be better understood by reference to the detailed description when considered in conjunction with the accompanying drawings, in which the components in the drawings are not necessarily to scale, emphasis instead being placed upon illustrating the principles of the invention, in which like reference numbers indicate corresponding parts throughout the different views.

[0011] [Figure 1] FIG. 1 is a diagram of an embodiment of an audiovisual environment configured in accordance with the systems and methods for immersive audio experiences of the present disclosure.

[0012] [Figure 2A] 1 is a first portion of a data flow diagram of an embodiment of how data may flow through an immersive audio system according to the present disclosure.

[0013] [Figure 2B] FIG. 2B is a second portion of the data flow diagram of FIG. 2A.

[0014] [Diagram 3]1 is a flowchart of an embodiment of a method for implementing an immersive audio system according to the present disclosure.

[0015] [Figure 4] FIG. 2 is a schematic diagram of elements of an exemplary computing device embodiment.

[0016] [Diagram 5] FIG. 2 is a schematic diagram of elements of an embodiment of a server-type computing device.

[0017] [Figure 6] FIG. 2 is a data flow diagram of another embodiment of an immersive audio system according to the present disclosure.

[0018] [Figure 7] FIG. 7 is a data flow diagram of an embodiment of an artificial intelligence (AI) engine as implemented in the immersive audio system of FIG.

[0019] [Figure 8] FIG. 7 is a data flow diagram of the Vibe Streaming Protocol as implemented in the immersive audio system of FIG.

[0020] [Figure 9] 1 is an embodiment of a lighting fixture according to the present disclosure.

[0021] [Figure 10A] 10 is another embodiment of the lighting fixture of FIG. 9 including four bulbs.

[0022] [Figure 10B] 10 is another embodiment of the lighting fixture of FIG. 9 including eight bulbs.

[0023] [Figure 10C] 10 is another embodiment of the lighting fixture of FIG. 9 including 16 bulbs.

[0024] Those skilled in the art will appreciate that elements in the drawings are shown for simplicity and clarity, and therefore not all connections and options are shown to avoid obscuring aspects of the invention. For example, common but well-understood elements that are useful or necessary in commercially feasible embodiments are less depicted to facilitate a more unobstructed view of these various embodiments of the present disclosure. It will be further appreciated that while certain actions and / or steps may be described or depicted in a particular sequence of occurrence, those skilled in the art will appreciate that such specificity to the sequence is not actually required. It will also be understood that the terms and expressions used herein are defined in relation to their corresponding respective fields of study and research, unless a specific meaning is otherwise set forth herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] The present invention will now be described more fully hereinafter with reference to the accompanying drawings, which form a part of this specification and which show, by way of illustration, specific exemplary embodiments in which the invention may be practiced. These exemplary and exemplary embodiments are presented with the understanding that the present disclosure is an illustration of the principles of one or more inventions and is not intended to limit any one of the inventions to the illustrated embodiments. The present invention may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein, but rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. In particular, the present invention may be embodied as a method or device. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Thus, the following detailed description is not to be taken in a limiting sense.

[0026] In some embodiments, the present disclosure describes systems and methods that may provide users with an immersive way to experience audio, including audio of music, podcasts, workouts, meditations, religious events, and the like. The system may include providing immersive experience software, which may be an application or other software operable on a user computing device that may receive music and other data inputs from one or more sources and output instructions to one or more audiovisual components within the user's home or other suitable location, such as a vehicle, office, gym, restaurant, etc. In some embodiments, the experience may simulate attending a live music event or may provide audio-based sensory input that provides a more immersive sensory experience than may be traditionally achievable at home. In some embodiments, the system may include interactive opportunities for artists and other users to more easily create custom lighting and media designs that may provide a unique and immersive listening experience.

[0027] Conventional systems that attempt to simulate a live music experience in the home or provide other sensory experiences related to music may use hardware devices that react based on the beats of the music as received through sensors / microphones, but these systems have little or no understanding of the song's tempo / BPM, mood, key, energy, beats, bars, phrases, call and response structure, instruments, vocals present, etc., and therefore do not accurately emulate something like a professional, custom-designed musical event, which may include synchronized and / or customized lighting designs.

[0028] To remedy this, in some embodiments, the system may include a software application that may, for example, receive an audio stream from a music streaming service or digital service provider (DSP) (e.g., Amazon Music, Apple Music, Spotify, YouTube®, etc.) or other audio source, overlay a digital data stream that may provide visual and other effects based on the characteristics of a particular song, and transmit the resulting data to an appropriate hardware device, such as, for example, smart LED lights, speakers, lasers, projectors, etc. In some embodiments, the data stream may include a stream type (e.g., song, podcast, workout, etc.), beats per minute (BPM), musical key, genre, artist, album, energy, etc. The stream may have a stream time code that may include a first beat, bars, phrases, scenes, and musical elements. Scenes may include intros, verses, choruses, buildups, drops, bridges, outros, etc. Musical elements may include, for example, drums, instruments, vocals, lyrics, melody, etc. In some embodiments, the system may operate with little or no additional input or musical skill required on the part of the user, such that an immersive experience may be achieved by virtually anyone. In some embodiments, the disclosed system may include hardware and / or software resources that may use artificial intelligence (AI), machine learning, and / or other digital processing techniques to create digital data streams to generate an improved audiovisual (A / V) environment for accessing music or other media.

[0029] A software application may use several data types and their interactions to generate the immersive effects described herein. In some embodiments, the primary data types may be music analysis, vibes, and instruments. Music analysis data types may include important features of a music track in a presentation-independent manner, including but not limited to both time and frequency domain characteristics, spectral analysis, chroma features, harmonic ratios, and Mel-frequency cepstral coefficients. Some presentation-related information may be included, such as describing the most important stems (e.g., vocal stems) in a track to help guide how they may be prioritized and assigned to instruments. Instrument data types may describe a collection of outputs and their addressable characteristics or capabilities. They could potentially be almost anything, but some categories include lights (e.g., Hue, Nanoleaf, LiFX, etc.), devices (e.g., scalp massagers, magic batons, gesture gloves, etc.), and visualizers (e.g., properties exposed by TV screens, VR headsets, software visualizers). In some embodiments, physical lights and / or devices may be grouped and assigned to zones within a media environment or space. The "vibe" data type may include hardware configuration properties, time-coded instructions / events, and other metadata. Vibes may encapsulate the creative output of a vibe creator (human or AI) and may serve to bind music analysis data to a set of instruments and properties along with their visual animations and representations (e.g., color transitions, instrument rotation, brightness over time). In some embodiments, vibes may be completely independent of both the music track (except for custom vibes) and the specific A / V instruments available to any given user or environment. In some embodiments, vibes may be universal or custom. Universal vibes may be designed to apply to virtually any music track, while custom vibes may be specific to a single track.In some embodiments, the custom tracks may include time-based triggers, animations, or other settings. A software application (i.e., a "client"), which may be a web application, a mobile application, etc., may interpret the vibe bindings based on the analysis data, adjust user preferences and / or overrides (e.g., different color schemes), and map the results to the instrument in substantially real-time. A user of the software application may also make custom adjustments such as color range (e.g., using a color wheel), intensity (e.g., adjusting the "contrast" of the experience, such as using more flashes (high contrast) instead of pulses (low contrast)), and / or brightness (e.g., the brightness of the overall experience).

[0030] In some embodiments, a method of using such a system may include separating drums, bass, lead, vocals, melody, instruments, etc. in an audio stream into separate channels and assigning each channel to a unique hardware device (e.g., drums to lasers and strobe lights, vocals to LED lights, etc.). In some embodiments, digital data streams that may control A / V hardware (e.g., smart lighting and / or experience devices, either real or virtual) may be provided by the actual artist who created / composed / performed the music or other audio, or from a sound / light designer working with the artist. For example, a musical artist may have used a particular set of lighting and other visual effects for a live concert in the past, and data for that performance may be converted into a data stream to control the hardware of the system in a similar manner. Alternatively (or additionally), the method may include creating and using data streams developed using machine learning or other artificial intelligence techniques. For example, machine learning may be used to analyze song data along with visualization data corresponding to the history and develop algorithms that may be applied to develop visualization data to control hardware on the fly, or may be stored and categorized by song or recorded performance.

[0031] In some embodiments, the system may be provided to the user as a kit with various levels of A / V hardware (either real or virtual) used with the software to control that equipment, or may be provided as pure software that may be compatible with different types of hardware devices that may exist or become available. In some embodiments, the user may implement the disclosed system and method using more or less robust A / V equipment or fixtures, ranging from one or two speakers or lights to more complex lighting, audio, haptic, and other sensory input equipment. The disclosed system may be flexible to scale the output based on the user's particular A / V settings, and may allow for scaling of that A / V setting if / when additional equipment may be provided.

[0032] Audiovisual environment FIG. 1 illustrates an exemplary embodiment of an A / V environment 50 in which the systems and methods disclosed herein may be implemented on behalf of a user 52. The A / V environment 50 may include various A / V equipment or fixtures 54, such as, for example, strobe lights 60, speakers 73, a TV monitor or other screen 62, ambient lighting 65, LED lights 70, lasers 72, 3-D and video projectors 75, virtual reality goggles 80, head massagers 83, spot and moving head lighting 85, etc. Those skilled in the art will recognize that additional types of A / V equipment may additionally be used within the scope of this disclosure, and that more simplified and / or complex lighting and sound systems may be used. Additionally, while the A / V environment may be substantially composed of real-world hardware, in some embodiments, the A / V environment may include all or substantially all virtual A / V equipment as may be present in a 3D simulated virtual environment. The A / V environment 50 may also include a user computing device 55, which may be a smartphone, tablet, laptop, desktop, or any other suitable computing device capable of running immersive experience software. In some embodiments, computing devices 55 and / or A / V equipment 54 may communicate with one or more remote cloud servers 100, either directly or through a home hub, such as home hub 90. In some embodiments, home hub 90 may be a computing device specially configured to communicate with A / V equipment 54, computing devices 55, and remote cloud servers 100. In some embodiments, home hub 90 may handle communications with A / V equipment and / or remote servers 100 on behalf of computing devices 55, or computing devices may communicate directly with A / V equipment and / or remote servers 100. Cloud server 100 may store and update a database of A / V data, which may correspond to a particular song, performance, audio event, etc., as previously created from, for example, analytics, machine learning, or other artificial intelligence techniques.For example, the remote cloud server 100 may contain a database of lighting and sound design data and files related to past concert performances that can be streamed or downloaded upon request to the computing device 55. In some embodiments, the cloud server may provide real-time analysis and on-the-fly generation of visual data for A / V equipment.

[0033] In some embodiments, the A / V equipment 54 may communicate with the computing device 55 individually via a wireless connection. In some embodiments, the A / V equipment 54 may communicate with the computing device 55 and / or the home hub 90 via a local area network (LAN), such as a common Wifi network or other wireless connection (e.g., Bluetooth, Near Field Communication (NFC), Radio Frequency (RF), 4G / 5G, 802.11a / b / g, etc.). In some embodiments, some or all of the A / V equipment 54 may include a wireless or wired connection to an A / V receiver 95, which may communicate with the computing device 55 to receive audio, video, and data files transmitted based on instructions from an immersive experience application. In some embodiments, the user computing device 55 may control the sound, video, audio, and lighting provided through the A / V equipment 54. In some embodiments, the A / V environment 50 may also include an auxiliary input / output device 90, which may be used to create physical gestures that may control the sound, video, audio, and lighting provided through the A / V equipment 54. In some embodiments, the auxiliary input / output device 90 may communicate with the computing device via Bluetooth or other wireless connection, or the computing device itself may function as the auxiliary input / output device. In some embodiments, the user may use the auxiliary input / output device 90 like a conductor's baton to control the intensity and type of audio and visual output generated by the A / V equipment 54. In some embodiments, the auxiliary input / output device 90 may be a Bluetooth baton with a three-axis gyroscope and infrared signals. The device 90 may be used to amplify or mute vocals, drums, or instrument layers, for example, to focus on a singer's voice, a guitar solo, or other particular portion of the music or other audio. In some embodiments, the user's body movements, dance, and / or hand gestures may be captured by a camera on the user's own equipment 55 or other hardware device to control the sound, video, audio, and lighting provided via the A / V equipment 54.

[0034] Data Flow and Processing As will become apparent from this disclosure, the immersive music systems described herein may be used in virtually any A / V environment, ranging from fairly minimal sound and visual systems to complex lighting and other special effects, or in any virtualized environment presented in a 3D virtual environment accessed via VR / AR hardware or other computing hardware. While the end experiences may be different in such environments, the systems and methods for processing the data flows and generating those experiences may be substantially similar or substantially the same in many aspects.

[0035] 2A and 2B are data flow diagrams illustrating embodiments of how data may flow through various components within an immersive audio system 200 as disclosed herein. In some embodiments, the data represented in the diagrams may be transmitted over any suitable network, such as, for example, the Internet, a cellular network, a local network, etc. Data flow may also occur within one or more computing devices, such as computing device 55 or cloud server 100, which may be connected over one or more networks. In some embodiments, the particular computers or servers that make up the computing environment used by the immersive audio system may be specially designed to run the immersive audio system or its components.

[0036] Vibe as referenced and used in the immersive audio system 200 disclosed herein may include multiple data types and data sets used by the system to function, including, but not limited to, experiential data containers including hardware / instrument configuration characteristics, time-coded instructions / events, and other metadata used in the immersive audio ecosystem. With reference to FIG. 2A, in some embodiments, the metadata and other data used by the system, and in particular the metadata extractor / importer 206, may be generated by obtaining data from one of a variety of data sources, such as artist-uploaded music 201, streaming services or DSPs 202, DJ software applications 203, live event streams 204, or artist or user-created vibe data 205. Of course, one skilled in the art will recognize that other audio / visual / media data sources may also be used consistent with the present disclosure. Data from each of these sources may flow to the metadata extractor / importer 206 to be processed.

[0037] The music 201 uploaded by artists may be stored in an audio library data store 219 hosted within the immersive audio system 200 ecosystem. In some embodiments, the immersive audio system 200 or immersive audio application may provide for artists to upload their music or other media to the audio library data store 219 so that the immersive audio system may generate vibe data based on their music. In some embodiments, the immersive audio system may include an application programming interface (API) specifically configured to provide for artists to upload music or other media to be used by the immersive audio system 200. Music and audio from virtually any audio streaming source or service 202, library, or content delivery system (CDN) may be analyzed to generate vibe data as well. In some embodiments, audio may be analyzed from one or more online streaming services, music databases, podcasts, other audio applications, etc., from which a user may choose to play audio selections. In some embodiments, this may be a streaming service 202, such as Spotify, Amazon Music, Apple Music, etc., or may be the user's own stored audio library. In some embodiments, audio stream data may be authenticated and metadata such as artist, audio type, song, genre, lyrics, album, API, etc. may be extracted from a streaming service or other audio source. In some embodiments, data may be received from a DJ software application platform 203 such as Pioneer Rekordbox. In some embodiments, data accessed from any source (201-205) may be sequenced lighting design data, digital multiplex (DMX) data, or other suitable forms of data.The data from the live event stream 204 may be for any of a variety of live event types, including audio, such as, for example, a music concert, sports or e-sports, workouts, plays, religious ceremonies, etc. In some embodiments, the live event data may reflect aspects of the light and sound design of the event, such as the desired atmosphere and emotion created by mixing lights, lasers, fog machines, fireworks, video screens, etc.

[0038] The audio from the audio library data store 219 or streaming service 202 may be provided in a form that can be digested by an immersive audio system or may be processed using an algorithm such as a vibe and logic (V+L) algorithm. In some embodiments, the V+L algorithm may be used to analyze the audio for virtually any number of songs and audio sources / streams to extract time-coded data. In some embodiments, various channels such as vocal channels, instrumental channels, and drum channels may be separated and analyzed individually or collectively. The extracted data may be stored in the vibe database 209, the audio library data store, or any suitable storage medium. For example, the V+L algorithm may extract musical and harmonic data such as, for example, track key, beats per minute (BPM), genre, time signature grid, song phrases and structure (intro, verse, build-up, drop, chorus, bridge, outro), frequency grid, intensity, energy level, etc. The V+L algorithm may include engaging one or more machine learning or AI engines. In some embodiments, and as described in more detail herein, the output of the V+L algorithm may be a data container, which may be vibration data.

[0039] In some embodiments, the stream data and light data from the DJ application platform 203 may be transformed, converted, cleaned, normalized, etc. The live event lighting design data 204 may be analyzed by an algorithm such as the V+L algorithm to ingest, clean, normalize, and convert the data into a consistent format that can be processed by the metadata extractor / importer 206. In some embodiments, key sequences or loops such as blends of specific colors, strobe patterns, lasers, spotlights, etc. may be saved as "scenes" for recall during the event, but also extracted by the V+L algorithm. In some embodiments, the Digital Multiplexing (DMX) protocol may be used to control A / V devices such as lights, lasers, and fog machines. In some embodiments, other protocols may be used to control A / V devices. In some embodiments, the V+L algorithm may analyze the DMX or other data and convert it into a consistent, usable format. In some embodiments, the live event data 204 analyzed by the V+L algorithm may be for a live event that corresponds to a song analyzed by the V+L algorithm, such that the live event data may correspond to the appropriate song data.

[0040] Vibe sequence data 205 created by a user or artist may be provided by the vibe and logic experience builder application. In some embodiments, vibe sequences for a particular artist, song, genre, performance, or other audio event may be created by an artist, fan, or other party. In some embodiments, a vibe sequence may be officially provided by an artist or other performer involved in the audio event. In some embodiments, the experience builder application may be used to create, save, and / or upload vibe sequences for an audio event. In some embodiments, each vibe sequence may provide a unique style of lighting or other sensory experience associated with a particular audio event, or associated with an artist, genre, or other category of audio performance.

[0041] The audio metadata extractor / importer 206 may receive data from an audio library data store 219, streaming services 202, DJ software or DMX data 203, and live event DMX streaming data 204. In some embodiments, the audio metadata extractor / importer may extract time-coded metadata, which may include, among others, beat grid, meter, segment, bar, beat, BPM, musical key, instruments, vocals, lyrics, transcript melody, drums, energy, song intro, build-up, chorus, drop, etc. In some embodiments, the audio metadata extractor / importer may separate vocal, instrument, and drum channels from an audio event in real-time. In some embodiments, the metadata extractor / importer 206 may include one or more AI or machine learning engines and may process and extract data as described in more detail in connection with FIG. 7 herein.

[0042] In some embodiments, the time-coded data from the song analysis performed by the metadata extractor / importer 206, and any normalized live event design data, may be stored in a raw labeled data store 207 as song analyzed audio data for use in machine learning or other artificial intelligence training and experience design. This data may alternatively or additionally be stored in a vibe database 209 on one or more servers, such as the remote cloud server 100, or in other suitable locations. In some embodiments, the song analyzed audio data may be curated by a vibe curator AI system 208. In some embodiments, the vibe curator AI system 208 may use machine learning techniques to determine visual patterns, color themes, motion and rhythm patterns, text, emojis, etc. that may be appropriate for a particular type of music or audio mood, music, genre, build-up, etc. The vibe curator AI system 208 may also incorporate song IDs, active devices, metadata, and lyric data into its analysis. In some embodiments, one goal of the VibeCurator AI system 208 may be to learn from historical live events and performances by DJs, sound designers, lighting designers, etc. how certain types of lighting and other effects can be applied to enhance a music or other audio experience. The VibeCurator AI system 208 may then be able to apply these techniques to virtually any song by analyzing the song or other audio and applying the appropriate effects. In some embodiments, the VibeCurator AI system 208 may predict and design a sensory immersive experience based on the audio stream content and the user's available A / V equipment or fixtures, whether real or virtual. In some embodiments, the VibeCurator AI system 208 may create "scenes" and loops, which may be composed of patterns, colors, themes, motions, rhythms, text, emojis, etc.The Vibucurator AI system 208 may then initiate a particular scene or loop via one or more of the A / V equipment as determined to be appropriate based on the particular audio event. In some embodiments, the scene or loop may be stored and transmitted between hardware components via a video file format. In some embodiments, the metadata extractor 206, raw labeled data store 207, and the Vibucurator AI system 208 may reside across multiple hardware and / or server components, such as the cloud server 100, including the A / V equipment components themselves.

[0043] In some embodiments, the vibe database 209 may receive output from the vibe curator AI system 208 and / or from vibe sequence data 205 created by an artist or user via the vibe+ logic experience builder application. In some embodiments, a "vibe" may be an experience data container that may include instrument mapping properties, time-coded instructions / events, and other metadata that may be used by the system to provide an immersive audio experience. In some embodiments, the vibe database 209 may store vibes, for example, on a remote cloud server, such as the remote cloud server 100 shown in FIG. 1. In some embodiments, the vibe database 209 may store multiple "vibes" per stream identifier (ID), such that an audio event, audio recording / stream, or part of an audio event or stream may have more than one option as a vibe. In some embodiments, vibes, universal vibes, vibe mixes, and other data used to provide an immersive audio experience may be reused across streams.

[0044] In some embodiments, the immersive audio system 200 may include or host a vibe marketplace 221 where vibes may be purchased, sold, traded, sampled, etc. In some embodiments, a user or artist who creates a vibe may wish to distribute it, sell it, or otherwise make it available to others. In some embodiments, vibe creators may decide to mint their vibes as non-fungible tokens (NFTs) 223 that may provide benefits to their holders. The vibe marketplace 221 may provide an exchange for artists to provide benefits to their fans through the sale of vibe NFTs, which may imbue holders with special privileges in either the real or virtual world. Those skilled in the art will understand that other uses of the vibe NFTs 223 or the vibe marketplace 221 may be contemplated consistent with the scope of the present disclosure.

[0045] In some embodiments, the vibe database 209 may interface with the vibe manager 210. In some embodiments, the vibe manager 210 may be hosted on a remote server, such as the remote cloud server 100, or on another server location. In some embodiments, the vibe manager 210 may provide authentication services to users accessing the vibe database 209 or other system features, for example, via the vibe+ logic player application. The vibe manager 210 may also provide device configuration, ID matching, device management, and provide access to scenes, stories, videos, audio types, and the like. The vibe manager 210 may also provide permission / access management and manage purchases and / or subscriptions for downloadable content (DLC) provided via the vibe database 209. In some embodiments, the vibe manager 210 may act as a gatekeeper to manage authorization, payment, and authentication when a user requests access to content on the vibe database 209.

[0046] In some embodiments, the stream manager 211 may manage access between a vibe and logic player application that may run on a user's computing device and one or more online streaming services, music databases, podcasts, other audio / video applications, etc., from which a user may select to play an audio, video, or game selection. In some embodiments, this may be a streaming service 202, such as, for example, Spotify, Amazon Music, Apple Music, etc. In some embodiments, the stream manager 211 may authenticate the audio stream data, and metadata, such as artist, audio type, song, genre, lyrics, album, API, etc., may be extracted from the streaming service or other audio source. In some embodiments, the stream manager 211 may also provide authentication services and ID matching for access to the streaming service 202. The stream manager 211 may also share data with the vibe manager 210 to coordinate which vibe in the vibe database 209 may best provide an immersive sensory experience for the selected audio stream. In some embodiments, the vibe and logic API gateway 212 may handle data communication between the user computing devices running the vibe and logic applications and / or the home hub.

[0047] 2B, data from the vibe manager 210 and stream manager 211 may be exchanged between user computing devices (e.g., computing device 55) running vibe and logic player application 213. In some embodiments, player application 213 may interpret or apply the received data and dynamically assign vocal, instrument, or drum channels to unique devices, such as A / V equipment 54. In some embodiments, player application 213 may separate vocal, instrument, and drum channels in real time, so that these channels may be analyzed or otherwise stored in light of data received from vibe database 209. It is contemplated that in some embodiments, this channel separation may be performed by vibe manager 210, or vibe curator AI 208, or metadata extractor 206, or on behalf of player application 213. Such channel assignments may allow each channel to be dynamically assigned to a unique device of A / V equipment 54; for example, drums may be assigned to strobe lights and lasers, vocals may be assigned to LED lighting, and instruments may be assigned to lasers. Of course, dynamic assignments may also vary based on audio data, such as genre, mood, etc., which may be determined by the vibe selected for a particular audio event. Thus, the application may synchronize channels, colors, patterns, and motion across A / V equipment 54, such as LED lights, strobe lights, spotlights, lasers, 2D or 3D projectors, augmented reality (AR) or virtual reality (VR) devices, smart monitors or displays, etc. In doing so, the player application may provide the user with an immersive music or other audio experience for virtually any song, either in the real world or in a virtual environment.In some embodiments, the player application interface may be managed via virtual reality or augmented reality using appropriate hardware, or via any other suitable media device that provides an interface between the user and the player application 213 platform.

[0048] In some embodiments, the system 200 may include a vibe and logic "conductor" hub 214 that may exchange data with the stream manager 211 and vibe manager 210, and may also provide instructions to the A / V devices 54. In some embodiments, the system may include a conductor hub (e.g., home hub 90 of FIG. 1) that may operate independently of the player application 213 on the user computing device, or some embodiments may not include a conductor hub at all, but instead communicate all data between the A / V devices and vibe manager 210 via the player application. In some embodiments, the user may access an interface via the player application 213 to select audio events, vibes, vibe previews, purchase DLC, etc., but once an audio event may be selected, the conductor hub 214 may actively manage to instruct the A / V devices 54 based on data exchanged with the steam manager 211 and vibe manager 210. Those skilled in the art may recognize that other separations of tasks may occur between the player application 213 and the conductor hub 214 while still being within the scope of this disclosure.

[0049] In some embodiments, the vibe manager 210 may recognize an audio event provided through the stream manager 211 as an event that has pre-analyzed audio data stored in the vibe database 209. In such an embodiment, the vibe manager 210 may apply the normalized audio event data for that audio selection to control the A / V equipment 54 accordingly. In some embodiments, if the vibe manager 210 cannot recognize the selected audio event as having stored audio data, the vibe manager 210 may apply visual patterns, color themes, motion and rhythm patterns, scenes, etc. learned from machine learning processes used by the vibe curator AI system 208. In some embodiments, the vibe manager 210 may receive metadata for the audio selection from the stream manager 211 and use it to determine which effect, theme, and / or vibe to initiate, and send instructions to the player application 213 and / or the conductor hub 214 accordingly. In such embodiments, the vibe manager 210, player application 213, etc. may provide substantially real-time generation of an immersive audio experience even for audio events that may not have been previously analyzed, or for audio events for which audio event data may not exist or may not be stored in the vibe database 209.

[0050] 3 is a flow chart of an embodiment of a method 300 for using an immersive audio system disclosed herein. At 302, the method may include initializing one or more A / V devices 54 in a user's system by a player application 213, a conductor hub 214, or a vibe manager 210. The initialization process may include establishing a connection with each A / V device 54, such as, for example, lights, displays, lasers, speakers, etc. In some embodiments, the player application 213 may include a process for discovering, adding, or removing A / V devices 54 from the application and / or a connection with the conductor hub 214. Initialization may also include determining what type of equipment each A / V device 54 may be, what its capabilities are, where it is located in the room, and how it is angled / rotated. In some embodiments, the application 213 may include a virtual map of the user's media viewing space, via a graphical user interface (GUI) and / or through augmented reality (AR), where the immersive audio system may be located and the user may be able to indicate the location of each device within the space. At 304, the method may include receiving audio event metadata from the audio source or from the vibe manager 210 via the vibe database 209. In some embodiments, the audio event may have been selected by the user or may have been selected from a playlist or any other suitable audio selection method. In some embodiments, the application 213 may include a graphical user interface (GUI) through which the user may make a song selection. In some embodiments, the interface for audio event selection may be provided via alternative means, such as, for example, VR, AR, voice commands, gestures, etc. In some embodiments, the selection of the audio event may be made via another application, such as, for example, a streaming service, a music player, a podcast player, etc.In some embodiments, audio event metadata may be received from a third party system, such as a streaming service (e.g., Spotify, Amazon Music, etc.). In some embodiments, a system such as the stream manager 211 may include one or more APIs that may provide communication between the player application 213, the vibe manager 210, and the third party streaming service application 202. In some embodiments, the player application and / or vibe manager may also supplement the audio stream with synchronized and / or time-coded experience metadata that may be instructed to the A / V device.

[0051] At 306, in some embodiments, the method may include determining whether vibe data may be stored in the vibe database 209 corresponding to the selected audio stream. For example, in some embodiments, the vibe data may be pre-extracted and generated in a manner such as that described above with reference to FIG. 2A. In such embodiments, the data may be stored in the vibe database 209 and retrievable by the vibe manager 210 when requested. If the vibe manager 210 or player application 213 determines at 308 that vibe data may be stored for the audio selection, the player application 213 and / or vibe manager 210 may retrieve at 310 the vibe data corresponding to the audio content or segment from the vibe database 209 or stored directly on the computing device 55.

[0052] If the application 213 or vibe manager 210 determines at 308 that vibe data is not stored for the selected audio or video content, then the application or stream manager 211 may analyze the audio content metadata at 312 to determine, for example, the artist, genre, tempo / BPM, lyrics, mood, key, energy, time signature, time signature grid, bars, phrases, sections, segments, tatum, call and response structure, instruments, and vocals present, as well as song sections such as intro, build, drop, chorus, outro, etc., frequency, spectral analysis, chroma characteristics, harmonic ratios, and mel-frequency cepstral coefficients, rhythm, intensity, and other song characteristics. In some embodiments, this analysis may be performed by a metadata extractor / importer 206 in communication with the vibe manager 210 and / or stream manager 211. At 314, the method may include separating the audio stream of the selected audio event into separate channels (stems), such as vocals, lead, drums, instruments, bass (either overall or individually), based on the song metadata. At 316, the system may generate vibe data from the audio event metadata using machine learning data, which may be stored in a vibe database. For example, as described above with reference to FIG. 2A, the VibeCurator AI system 208 may compile data based on historical live events to determine optimal use of visual patterns, color themes, motion and rhythmic patterns, etc. The VibeCurator AI system 208 may apply this learned information to the audio event metadata of the selected song to generate visual or sensory experience data for the selected audio event. It is contemplated that in some embodiments, the method may include applying the AI ​​system data to all audio selections instead of referencing stored audio event data.

[0053] At 318, the method may include allocating the separated audio channels to individual portions or groups of A / V equipment 54, which may include using the generated vibe data from 316 and the normalized vibe data from 310. In some embodiments, an analysis of the characteristics and metadata for each individual channel is performed to generate visual or sensory experience data that may be applied based on the operation of that particular channel. For example, LED lights may be assigned to operate based on vocal data, lasers may be assigned to operate based on instrument data, and spotlights may be assigned to operate based on drum data. Those skilled in the art will appreciate that other combinations of assignments may also be generated within the scope of this disclosure. In some embodiments, the determination of the channels to assign to each type of A / V equipment may be based on a static algorithm or may be based on information learned from the VibeCurator AI system 208 during machine learning based on historical audio events. At 320, the method may include sending instructions to the A / V equipment 54 based on the visual or sensory experience data to create an immersive audio and visual environment that moves, changes, and reacts throughout the playback of a selected song or audio stream, such as podcasts, sports and e-sports, games, movies, etc.

[0054] FIG. 4 is a simplified diagram of some physical elements that may make up an embodiment of a computing device, such as computing device 55, and FIG. 5 is a simplified diagram of physical elements that make up an embodiment of a server-type computing device, such as may be used for remote cloud server 100. Referring to FIG. 4, a sample computing device physically configured to be part of a system and method for an immersive audio experience is shown. Computing device 55 may have a processor 1451 physically configured according to computer executable instructions. In some embodiments, the processor may be specially designed or configured to optimize communication between servers associated with the systems described herein. Computing device 55 may have a portable power source 1455, such as a battery, which may be rechargeable. The computing device may also have a sound and video module 1461 that assists in displaying video and sound, and may be turned off when not in use to conserve power and battery life. Computing device 55 may also have volatile memory 1465 and non-volatile memory 1471. Computing device 55 may have GPS capabilities, which may be a separate circuit or may be part of processor 1451. There may also be an input / output bus 1475 that sends and receives data to various user input / output devices, such as a microphone, camera, display, or other input / output devices. Computing device 55 may also control communication with a network via wireless or wired devices. Of course, this is just one embodiment of computing device 55, and the number and types of computing devices 55 are limited only by the imagination.

[0055] The physical elements that make up an embodiment of a server, remote cloud server 100, are further illustrated in FIG. 5. In some embodiments, the server may be specially configured to execute the systems and methods for immersive audio experiences as disclosed herein. At a high level, the server may include digital storage, such as, for example, magnetic disks, optical disks, flash storage, non-volatile storage, etc. The structured data may be stored in a digital store in a database. More specifically, the server 100 may have a processor 1500 that is physically configured according to computer executable instructions. In some embodiments, the processor 1500 may be specially designed or configured to optimize communication between a computing device, such as computing device 55, and A / V equipment or remote cloud server 100 as described herein. The server may also have a sound and video module 1505 that assists in displaying video and sound, and may be turned off when not in use to conserve power and battery life. The server 100 may also have a volatile memory 1510 and a non-volatile memory 1515.

[0056] A database 1525 for digitally storing the structured data may be stored in memory 1510 or 1515 or may be separate. The database 1525 may also be part of a cloud of servers and stored in a distributed manner across multiple servers. There may also be an input / output bus 1520 that sends and receives data to various user input devices, such as microphones, cameras, display monitors or screens. The input / output bus 1520 may also control communication with the network, either via wireless or wired devices. In some embodiments, an immersive experience controller for executing the immersive experience API may be located on the computing device 55. However, in other embodiments, the immersive experience controller may be located on the server 100, or on both the computing device 55 and the server 100. Of course, this is just one embodiment of the server 100, and additional types of servers are contemplated herein.

[0057] Vibe Creation System FIG. 6 is a diagram of another embodiment of an immersive audio system 600. The system 600 may process audio or other media data to provide vibe data in a data format that may be implemented in various audiovisual and / or media environments to provide an immersive multimedia experience. The system 600 may retrieve music or other media from one or more music sources 601, which may include a music content delivery network (CDN), a Vibes+Logic master vault 604, music and / or podcast NFTs 606, and a music streaming DSP 608. The music CDN 602 may contain music by unsigned, unlabeled artists who may upload their own music or other media. The master vault 604 may contain track remixes that may be created specifically for visualization purposes, such as using visualizations generated through the immersive audio system 600. The music and / or podcast NFTs may be music tracks or other media minted as NFTs by an artist or another entity. The music streaming DSP 608 may be any of a variety of music streaming DSPs, such as, for example, Amazon Music, Apple Music, Spotify, YouTube®, SoundCloud, Tidal, etc.

[0058] In some embodiments, a music track from one or more music sources 601 may be selected for processing and generating vibe data, such as by a user accessing a client software application running on a user computing device, such as a desktop computer, mobile device, tablet, etc. In some embodiments, the client software application may be executed via a virtual reality (VR) or augmented reality (AR) (VR / AR) device, such as VR goggles. The selected track may be processed by AI analysis 610, which may take a fairly flat music track and, through analysis, extract various types of data related to the track. For example, AI analysis 610 may determine information related to the track, such as, for example, beats per minute (BPM), frequency in various parts of the track, location and / or timing of different sections of the track (e.g., chorus, verse, build, drop, etc.), key transition moments, stems, etc. In some embodiments, some of this information (e.g., stems) may be provided by the artist, such as by uploading to the master vault 604. Determining this information may enable the system to better understand the components of a song and make better predictions regarding transitions across different song portions that may benefit from different visualizations.

[0059] In some embodiments, the system 600 may provide for generating visual accompaniment to a music track based on song information extracted by the AI ​​analysis 610. The visual accompaniment may be generated using the AI ​​vibe designer 612, the universal vibe mix engine 614, the vibe creator 616, or a combination thereof. In some embodiments, the AI ​​vibe designer 612 may interpret data generated in the AI ​​analysis 610 to determine a track-specific visual accompaniment for a song. The visual accompaniment may include determining appropriate instructions / animations for A / V equipment for different parts of a song based on the song data. For example, based on machine learning and historical data, the AI ​​vibe designer 612 may determine that certain musical aspects of a song may be paired with certain visualizations, such as, for example, certain bright colors for drum beats, strobe lights for vocals, etc. Those skilled in the art will recognize that these are merely examples of possible visualization mappings, and that the potential variety of the output of such a mapping process may be virtually unlimited. As a result, the AI ​​vibe designer 612 may create a song-specific visual mapping that may result in a generative visual accompaniment. One embodiment of a process that the AI ​​Analytics 610 and AI Vibe Designer 612 may perform in extracting data and generating visual accompaniment is described in more detail with respect to FIG.

[0060] Non-track specific visual accompaniments may also or alternatively be provided via the Universal Vibe Mix 614. In some embodiments, the Universal Vibe Mix 614 may use track data from the AI ​​Analytics 610 to predict lighting instructions, animations, and other visual effects to be assigned to specific sections of each song regardless of the particular song. In some embodiments, the assigned lighting and other visual effects may be pre-created and pre-existing by a user or one or more AI engines. For example, music tracks may each include similar sections, such as an intro, verses, pre-chorus, chorus, bridge, break, build, drop, etc. In some embodiments, the Universal Vibe Mix 614 may assign the same pre-existing lighting instructions, animations, or other visual design accompaniments to the chorus of each song regardless of which song is selected or each build, drop, etc. Thus, in the Universal Vibe Mix 614, music tracks may be accompanied by immersive visualizations and vibe data mapped to different sections of a song, but the particular visualizations used may not be specific to that song. Thus, while the universal vibe mix 614 may not be fully tailored for a particular song, the visualization information may still be mapped to the characteristics of the song as determined by the AI ​​analysis 610 such that it may provide a synchronized visual accompaniment for an immersive user experience. In some embodiments, the universal vibe mix 612 may be used as a first-time vibe editor for new song sections for which the user or AI has not yet created a tailored vibe mix. In some embodiments, users may create universal vibe mixes and make them available for purchase, sharing, or distribution to other users, either as NFTs or distributed more broadly.

[0061] In some embodiments, the system 600 may include a vibe creator 616, which may be accessed via a client software application. The vibe creator 616 may receive track data extracted and / or generated by the AI ​​analysis 610, the vibe designer 612, and / or the universal vibe mix 614. A user (e.g., an artist, a fan, etc.) may access the vibe creator 616, for example, to create a user vibe mix 618. In some embodiments, the vibe creator 616 has a user interface for assigning and adjusting lighting instructions, animations, and other visual accompaniments, labels for different time-coded sections of audio or audio sections (e.g., choruses), and hardware instrument mappings. In some embodiments, the user vibe mix 618 may result from a user further customizing an AI-generated vibe mix from the AI ​​vibe designer 612, or the user may create a completely custom user vibe mix with or without data from the AI ​​analysis 610. In some embodiments, the vibe creator 616 receives information from the universal vibe mix engine 614, which may enable a user to edit the universal vibe mixes (either user-generated or AI-generated), further customize those mixes to specific tracks, or generate additional universal vibe mixes that may be applied to various tracks. In such embodiments, a user may work from scratch or add to an AI-generated vibe mix to create a unique vibe mix for a specific track that may reflect a unique immersive audio experience. In some embodiments, the AI ​​vibe designer 612 may analyze the user vibe mixes created in the vibe creator 616 to further improve its AI capabilities.In other words, a user creating a custom vibe mix for a particular track may be part of an iterative machine learning process in which the AI ​​vibe designer 612 generates track-specific vibe mixes based on AI analysis 610, and may then update and refine those vibe mixes based on user-created vibe mixes 618 by one or more users. In this manner, the AI ​​vibe designer 612 may improve its AI-generated vibe mixes.

[0062] In some embodiments, the system 600 may include a minting engine 620 that may provide for minting the user vibe mix 618 on one or more blockchains. The resulting minted vibe mix may be a user vibe mix NFT or other digital asset that may be owned in whole or in part by the user. The minted vibe mix may incorporate digital contracts that provide for certain treatment related to ownership shares, royalties, profit sharing, resale, underlying intellectual property (IP), and the like. In some embodiments, the system 600 may include a vibe marketplace 622 where users may sell, exchange, or otherwise distribute the user vibe mix 618 and / or user vibe mix NFTs. The vibe marketplace 622 may be accessed, for example, via a client software application, via a web browser, other third party software application, and the like. In some embodiments, the vibe marketplace 622 may provide samples for potential purchasers to "try" the user vibe mix 618 before purchasing. In some embodiments, the vibe marketplace 622 may be accessible as a virtual store or shop via the Internet, the metaverse, a VR / AR application, or a client software application. In such embodiments, a user may virtually "enter" the vibe marketplace 622 and virtually purchase user vibe mixes that may be created for particular tracks by particular artists, genres, moods, etc. Those vibe mixes may then be played or experienced by the user through either real-world A / V fixtures or virtual A / V fixtures such as virtual 3D environments.

[0063] In some embodiments, the system 600 may also include an account gateway 624. The account gateway 624 may verify that a user creating, requesting, accessing, or purchasing vibe mixes or other information via the immersive audio system 600 is authorized to do so with the specific user information provided, such as a specific user account. In some embodiments, a user may log into a specific user account using verifiable user credentials (e.g., login, password, two-factor authentication, etc.). Each user account may have access to different resources, such as previously created or purchased vibe mixes, vibe creation projects, etc. The account gateway 624 may be a gatekeeper that verifies this authorization information and provides the correct user with access to specific resources within the system 600. The account gateway 624 may also process subscription information, such as monthly and / or yearly subscriptions to services provided via the system 600 or access to vibe mixes on the system. The account gateway 624 may also provide for processing payment information for purchases or subscription fees processed through the system 600.

[0064] In some embodiments, the system 600 may deliver the content of the vibe mix using a vibe streaming protocol 626, an embodiment of which is described in further detail in FIG. 8. The vibe streaming protocol 626 may provide a streaming protocol that includes both audio and visual data bound together in a single stream, including visual accompaniment to the audio. In this manner, the visual beat-based information developed by the system 600 through the vibe mix may be combined and delivered together to applicable hardware or A / V equipment. In some embodiments, the vibe streaming protocol 626 may be accessed via a web browser, via smart home lighting fixtures, VR / AR applications, etc. One embodiment of the vibe streaming protocol 626 is described in further detail below with respect to data formats.

[0065] The vibe streaming protocol 626 may be streamed over the Internet or other network to any of a variety of user A / V equipment, such as VR / AR / XR goggles 628, visualization or other video monitors 630 (e.g., TVs and monitors), and smart fixtures 632 (e.g., smart lighting, Internet of Things (IoT)). In some embodiments, the vibe streaming protocol 626 may be accessed and utilized via actual hardware components (e.g., IoT lighting, AR / VR / XR devices, monitors, etc.) or by virtual fixtures in a virtual space (e.g., the metaverse). For example, in some embodiments, a user may access a space in the metaverse that has a particular virtual media setup, such as a virtual sound system or visualization features, that may or may not have a real analog. Alternatively, an artist may hold a concert or show in a virtual 3D theater, which may include virtual visualizations accessible via AR / VR / XR devices or other features. For either virtual or real hardware, the vibe streaming protocol 626 can be designed to be accessible by either, or in combination, to provide an immersive audio experience to users and others.

[0066] Data Format In some embodiments, the system 600 may include several types of data, including analytical data, instrument data, scene data, and vibe data. In some embodiments, the analytical data may be track data generated by the AI ​​analysis 610 and may describe important characteristics of the music track in a presentation-independent manner. Some presentation related information may be included, such as describing the most important stems in the track to help guide how they are assigned to instruments. In some embodiments, the format may be modular and decomposable, meaning that an entire track may be sent as a single file, or individual objects (e.g., "bars") may be streamed along with the music. In some embodiments, the format may also be deterministic, meaning that it may be relatively easy to request or locate the correct data for any point in the track (e.g., if a user asks for a random time code).

[0067] In some embodiments, the analysis data may include set objects, track objects, metadata objects, feature objects, section objects, and stem objects. Set objects may allow a single audio file / source to expose multiple tracks. Track objects may be containers for sections. Metadata objects may hold set or track data that may not have been generated as part of the AI ​​analysis 610. Feature objects may include a collection of features that may describe a particular musical selection. This can be an entire track, a section, an individual bar, etc. Section objects may represent a period within a track. Section objects may be segmented based on significant changes to stems, but may also be used to pass information about smaller "chunks" along with streaming events, for example. In some embodiments, section objects may be the primary transport for analysis data. Each section object may convey important information about the mood and energy of that portion of the track, and may include a collection of stems that divide the section into its constituent audio "layers," and suggestions for mapping the stems to A / V instruments. In some embodiments, the client software application may combine this data with information in the vibe and user settings to determine how to map the properties and stems to instruments and instrument parameters. Stem objects may represent individual instruments or voices in the music.

[0068] Fixture data may describe a collection of outputs, such as real or virtual A / V devices, and their addressable characteristics or capabilities. An output could potentially be almost anything, but some categories could be lights (e.g., LED strips, Hue, Nanoleaf, LiFX, etc.), devices (e.g., scalp massagers, magic baton, gesture gloves, etc.), screens (projectors, TVs, tablets, virtual walls (in VR / AR / metaverse), smartphones), and visualizers (e.g., properties exposed by a software visualizer). Physical or virtual lights and devices may be grouped by the user and assigned to zones, such as grouping several specific smart light bulbs together and assigning them to a "mood." The fixture data format may accomplish two main goals: (1) to define each feature, its function, and any user settings that may change those functions, and (2) to assign features to known channels that can be logically bound to analytics data via vibes.

[0069] Scene data may include stage objects, fixture objects, and channel objects, and may include fixture controllers. Stage objects may encapsulate all data related to a particular user's complete A / V fixture setup and may be primarily a container for "fixture" objects. Fixture objects may define the type, function, and settings of a single fixture. Each fixture may be of a known type with a defined set of expected configuration properties. In some embodiments, fixtures of type "composite" may allow multiple fixtures to be exposed by a single device. For example, a software visualizer may have multiple lighting fixtures in addition to other lighting fixtures. Channel objects may associate fixtures based on user assignments so they can be controlled as a group with specific behaviors defined in the vibe.

[0070] The vibe data may be the output of the AI ​​Vibe Designer 612, Universal Vibe Mix 614, or Vibe Creator 616, or a "vibe mix" or "vibe." The vibe data may encapsulate the creative output of the vibe creator (human user or AI) and may act as a glue that binds the music analysis data (extracted by the AI ​​Analysis 610) to a set of instruments and characteristics that may occur in any given A / V setting. The vibes, especially from the Universal Vibe Mix, may be completely independent of both the track and the specific A / V instruments or equipment that may be available to any particular user. The vibes themselves may contain information about a predefined number of zones, such as zones 1-8, although each vibe may contain zone mappings that may be implemented later according to a particular user's A / V settings and zone assignments. The vibe data may include binding objects and other objects that together define the relationship between the music analysis data and the instruments or similar objects. In some embodiments, this may be composed of vibe objects, rule objects, time offset objects, and binding objects. Those skilled in the art will recognize that these are merely examples of possible bindings, mappings, and relationships between audio and visual mappings, and that the potential variety of objects for such a mapping process can be virtually limitless.

[0071] In some embodiments, at its core, a vibe object may be a container for a collection of "rule" objects that may define "bindings" between stem features and A / V instrument functions, and instructions for when those bindings may be applied. Additionally, a rule object may associate one or more output bindings with one or more time offsets. In other words, a rule object may connect the "what" with the "when". For example, a rule object may include instructions for applying a particular binding (e.g., lighting manipulation, visualization design, etc.) to a particular section (which may be genre-specific, for example) of a particular type of track, such as the use of dim pulsed lighting for the intro of an electronic dance music (EDM) song. A rule object may also specify track-specific filters. A track may include any kind of song section, which may be updated or developed over time. For example, some track sections may include intros, verses, pre-choruses, choruses, bridges, outros, breaks, builds, drops, etc. Some sections may be specific to different genres, while other sections may be used in virtually any genre. Other rule objects could include specific changes in frequency prominence or coverage, key shifts, vocal prominence, or bindings to any number of time-coded events identified by the AI ​​analysis 610 (e.g., lighting manipulation, visualization designs, etc.) One skilled in the art will recognize that these are merely examples of possible rule objects and that the potential variety of rules around such processes can be virtually limitless.

[0072] A time offset object may be a reference to a section. It may be specified in time (e.g., milliseconds), bars, or beats. A time offset object may include any combination of "start", "end" and "length". In some embodiments, the start and end may "pin" a timing range to the start and end of the section. The length may specify a fixed time extending forward from the start, or a fixed time extending backward from the end. In some embodiments, negative values ​​of the start and end values ​​may potentially be used to extend the outer range of a selected section. A binding object may connect a single feature of a stem (or a smart stem, a dynamic assignment of stems, e.g., "front" may be a smart stem that indicates which stem is most prominent or dominant to the human ear perception) to one or more features of a fixed zone. In some embodiments, each feature type may have a different set of parameters.

[0073] Vibe Streaming Protocol In some embodiments, the vibe streaming protocol 626 may provide a streaming protocol that includes both audio and visual data bound together in a single stream, including visual accompaniment to the audio. In this manner, the visual, beat-based information developed by the system 600 through the vibe mix, either real or virtual, may be combined and delivered together to applicable hardware or A / V equipment. In some embodiments, the vibe streaming protocol 626 may include vibe data encoded directly into the audio stream, which may be in a unique file format. The vibe data included in the stream may be the same or similar data described above with reference to "Data Formats," and visualization data for a particular section or moment of a song may be included directly in the data stream along with the audio. The vibe streaming protocol 626 may enable streaming devices to access the accompanying visual data in sync with each segment of the audio stream. In some embodiments, the unique file format may be Vibes JSON Grid, or VJG, or "Grid."

[0074] FIG. 8 is a diagram 800 of an embodiment using HTTP Live Streaming (HLS) with vibe streaming protocol. The diagram includes an input 802, a server 804, a distribution 806, and a client 808. The input may be a music source 810, as described in more detail with reference to FIG. 6. The server 804 may be one or more computer servers or cloud servers operating an immersive audio system such as the immersive audio system 600, and / or may be an HLS server. In some embodiments, the server 804 may include a media encoder 812 that may encode the visualization and audio portions of the vibe mix. In some embodiments, this may be the same as or similar to the methods described herein, as implemented by the AI ​​analysis 610 and the AI ​​vibe generator 612. The resulting media file may include both the audio data and the visualization data included in the vibe mix as generated. In some embodiments, such a file may be a VJG file type, and may be combined with an MP3 file that includes the audio data. The stream segmenter 614 may split the VJG and MP3 files into FVJG and FMP3 files into segments that can be streamed. In some embodiments, the origin web server 816 may distribute the files over the network 822. An index file 818 may reference different segments of the stream, each segment containing the appropriate visualization and audio data for that portion of the song. A particular segment may be an f file 820 that may be delivered in segments over the network 822 to a client 824. The client 824 may be a computing device that may run a client software application as described herein that may interpret the user's A / V instruments and assign visualization cues to them. Thus, the vibe streaming protocol may include instructions for what may occur visually over a particular segment of the stream, thereby distributing visual accompaniment along with the audio.

[0075] In some embodiments, the same vibe streaming protocol may be used, which may include zone information for zones beyond the user's capabilities, regardless of how many zones a user may configure for his or her particular media environment. For example, each vibe streaming protocol file may include information available for up to eight zones, although other maximum numbers of zones may be used. When a client software application requests a vibe using the vibe streaming protocol, it may only request information regarding the number of zones the user's equipment can accommodate. The vibe streaming protocol may be used regardless of the user's particular media environment, because the visual information in the vibe mix may be mapped by zone, and the client software application separates the A / V instruments into zones.

[0076] AI engine analysis FIG. 7 is a data flow diagram 700 depicting the data flow of one or more AI engines for analyzing generated vibe mix data as represented by AI Analytics 610 and AI Vibe Designer 612 in an immersive audio system 600. Data from one or more music sources 601 may be received by AI Analytics 610. AI Analytics 610 may include one or more AI engines, such as a first AI engine 650 and a second AI engine 652. The first and second AI engines 650, 652 may perform track analysis to convert relatively flat audio files into volumes of data that may be used to locate portions of songs that may be treated differently for visualization purposes. The music sources may provide music data in one or various formats, such as MP3. Music tracks may include certain metadata, such as artist, title, genre, length, etc. The first AI engine 650 may analyze the track to determine song characteristics, such as beats per minute (BPM), beat, section, etc. In some embodiments, the first AI engine 650 may send track metadata to a DSP API 670, such as Spotify, to return this track information. The first AI engine 650 may also analyze the track to determine stem MP3 and stem PCM information. The stem PCM may include stem data associated with specific drums (e.g., snare, kick, bass), vocals, lead / melody, etc. The second AI engine 652 may receive the stem data, BPM, time signature, section, and genre data and generated phrases. In some embodiments, the phrases may include the start and / or end of a particular section (e.g., drop: start and end, verse: start and end). Thus, the second AI engine 652 may locate and generate data indicating where such phrase events occur throughout the track.

[0077] The AI ​​vibe designer 612 may include one or more AI engines that may receive the analysis data from the AI ​​analysis 610 and apply appropriate visualizations that may be used to create an immersive audio experience. The embodiment of FIG. 7 includes a third AI engine 654 and a fourth AI engine 656. In some embodiments, the third AI engine 654 may receive stem PCM data generated by the first AI engine 650, phrase data generated by the second AI engine 652, and BPM information received by either the DSP API 670 or the first AI engine. The third AI engine 654 may process this data to generate vibes, vibe mixes, or vibe data. Vibes may be time-coded instructions and events of the experience data container, as well as other metadata. For example, in a song that may include several 8-bar musical phrases, the generated vibes may enrich the audio file with several 8-bar long vibes. In some embodiments, the vibes may instead be the length of the phrases. In some embodiments, a single audio may include many compatible vibes. In some embodiments, the vibes generated by the third AI engine 654 may include data fields such as name, mood, musical key, phrase energy / intensity, level / loudness, curves (e.g., stems changing over time), etc. In some embodiments, the moods may include different tones such as confused, sad, strong, happy, angry, cheerful, compassionate, depressed, scared, hurt, lonely, regretful, chill, etc., which may be represented by different visualization effects. In some embodiments, the first AI engine 650, the second AI engine 652, the third AI engine 654, and the fourth AI engine 656 may operate together in an iterative manner to improve each of their functions as described herein. Furthermore, one skilled in the art will recognize that the order of operations does not necessarily follow the same order as described, and that different or combined AI engines may analyze and extract similar data while still being within the scope of the present disclosure.

[0078] In some embodiments, the fourth AI engine 656 may receive the vibe data generated by the third AI engine 654 and device configuration data that may be specific to a particular type of equipment. For example, a rotating head instrument may have specific configuration specifications related to performance and capabilities that the fourth AI engine 656 may process to assign a particular activity to that instrument. In some embodiments, the device configuration data received by the fourth AI engine 656 may be for a particular user's A / V instrument environment or may be for a more general category or type of instrument. The fourth AI engine 656 may process the vibe data and the device configuration data to generate sequences. Each sequence may assign a particular task to a particular A / V instrument or instrument type to be activated at a particular portion of the track to fulfill the generated vibe experience. For example, a sequence may include information for a rotating head device to operate for a particular length of time, which may be the length of a phrase or a particular vibe. A sequence may also include a particular action for the device to perform and when to perform with reference to a particular sequence. For example, a 48 second sequence may "start" at 0 seconds and end at 48 seconds with reference to the sequence. During those 48 seconds, the sequence data may include information related to the rotation (X, Y, and Z axes), color, aperture, and start time of a particular action. For example, the sequence may include data for a rotating head to rotate to a 180 degree position on the Y axis within 24 seconds of the sequence and back to the 0 degree position on the Y axis at the 48 second mark. Those skilled in the art will appreciate that detailed sequences may be generated for each phrase and / or vibe of each type of A / V instrument in a track.

[0079] In some embodiments, the output of the AI ​​data flow diagram 700 may be a vibe mix 658, as shown and described in more detail herein. The vibe mix 658 may include data that may be bound to an audio track to provide a particular immersive audio experience via the user's real or virtual A / V equipment. For example, in some embodiments, the vibe mix 658 may include information such as, for example, the name, track (in the case of music track specific vibes), and names of the vibes included within the vibe mix, as well as where those vibes should start and end (either via beats or units of time), and where within the track in a particular sequence. In other words, the vibe mix 658 may include the data necessary to create the desired immersive audio experience provided by the system 600.

[0080] Real and Virtual Lighting Fixtures As will be appreciated by those skilled in the art, the vibe data generated by the immersive audio system 600 may be implemented by a variety of different real or virtual A / V fixtures, including lighting, smart lighting, visualization environments, lasers, and the like. FIG. 9 is an embodiment of a custom smart lighting fixture 900 that may be used in the context of the immersive audio system described herein or in other media environments. The lighting fixture 900 may be substantially circular or ring-like in shape, may be configured to mount to a wall, ceiling, or other surface in a given media environment, and may be powered via battery, A / C power, and the like. In some embodiments, elements of the lighting fixture 900 may be configured to connect to a wireless network via wifi, Bluetooth, or other wireless or wired connections. In some embodiments, the lighting fixture 900 may combine mirrors, colored bulbs, LED lights, light-refractive materials, and may be synchronized to music via software, such as a client software application described herein, or via audio sensing capabilities or other software control. The lighting fixture 900 may include multiple light bulbs 902, which may be colored and may be low-latency smart bulbs that are configurable to sync to music. FIG. 9 shows a lighting fixture embodiment 900 including eight bulbs 902, although the number of bulbs 902 may vary from embodiment to embodiment and depending on the desired visual effect. Light fixture 900 may also include light-refracting elements 904, which may include reflective textures to create lighting effects. In some embodiments, the refractive elements 904 may be arranged in a ring that substantially covers each bulb, as best seen in FIGS. 10A-10C. From the observer's perspective, the bulbs 902 may be behind the refractive elements 904 such that the light of the bulbs may shine through the refractive elements into the room, creating a visual effect. In some embodiments, the refractive elements 904 may be constructed of one or more pieces, may be substantially transparent or translucent, may be vivid in color, or may be tinted in one or more colors.

[0081] In some embodiments, the lighting fixture 900 may include one or more LED strips 906. The LED strips 906 may be gradient multi-zone LED strips, including infrared, that may be wrapped outwardly on the inside (bulb side) of the refractive element 904 to shine light through the refractive element. In some embodiments, the LED strips 906 may be arranged to generally form substantially an LED ring, which may trace the ring pattern established by the bulb 902 and the refractive element 904. The LED strips 906 may also be synchronized to music. In some embodiments, the lighting fixture 900 may also include one or more mirrors 908, which may be mounted on the back of the fixture such that the bulb 902 may be positioned between the mirrors 908 and the refractive element 904. In some embodiments, the mirrors 908 may extend outward beyond the boundaries of the ring of the bulb 902 and the refractive element 904. The mirrors 908 may reflect the light of the bulb 902 and the LED strips 906 and amplify their effect.

[0082] In some embodiments, the LED strip may be divided into LED zones. For example, FIG. 10A shows an embodiment of a lighting fixture 900a including four bulbs 902a arranged substantially evenly around a ring-shaped refractive element 904a. The lighting fixture 900a also includes an LED strip 906a arranged in a ring around the refractive element 904a and divided into four LED zones 906a1, 906a2, 906a3, 906a4. Each of the four zones may be substantially disposed between a respective adjacent bulb 902a. The LED strip 906a may be programmed or instructed to show a different color of illumination for each zone in coordination with the adjacent bulbs 902a or other zones. FIG. 10B shows a similar embodiment of a lighting fixture 900b including eight bulbs 902b, a mirror 908b, a refractive element 904b, and an LED strip 906b, which may be divided into eight zones. Similar to FIG. 10A, the LED strip 906b of FIG. 10B may include separate zones defined between adjacent bulbs, e.g., 8 different LED zones. FIG. 10C shows a similar embodiment of a lighting fixture 900c including 16 bulbs 902c, a mirror 908c, a refractive element 904c, and an LED strip 906c that may be divided into 8 zones. The LED strip 906c of FIG. 10C may include separate zones defined between adjacent bulbs 902c, e.g., 16 different LED zones. In some embodiments, it may be desirable for the lighting fixture 900 to include a number of lights and / or corresponding LED zones that may be divisible by a number of four or a number of two. In some embodiments, any desired number of lights and corresponding LED zones may be used to convey a desired visual effect.

[0083] In some embodiments, virtual lighting "fixtures" may be implemented in conjunction with the immersive audio system disclosed herein. In some embodiments, a user may create, purchase, or otherwise acquire a digital avatar or image, which may include layers that may be music-responsive. The digital avatar may act as the user's "digital twin," which may be used throughout the metaverse to attend concerts, buy vibes, or otherwise interact with other avatars. In some embodiments, the digital image may be minted as an NFT, which may be unique to a particular user or owner of the NFT. During the minting process, the avatar image may be separated into various PNG layers and assembled into various variations. After minting, all layers may be mapped as if they were virtual lighting fixtures. In other words, a user's digital twin may include different layers, each of which responds to music in a unique way. In some embodiments, the layers may process and react to the vibe mix generated using the immersive audio system described herein, or the user may design their own vibes for the digital avatar to respond (e.g., as a fixture) according to the mapping of the avatar's layers. In some embodiments, a user's digital avatar or NFT avatar may be seen by other users in the metaverse with a musically responsive layer that creates a unique visual display, such as through a visualized output of one or more vibe mixes as disclosed herein.

[0084] The figures depict preferred embodiments for illustrative purposes only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods shown herein may be employed without departing from the principles described herein.

[0085] Upon reading this disclosure, those skilled in the art will appreciate still further alternative structural and functional designs for the systems and methods described herein through the principles disclosed herein. Thus, while specific embodiments and applications have been illustrated and described, it should be understood that the disclosed embodiments are not limited to the precise structures and components disclosed herein. Various modifications, changes and variations that will be apparent to those skilled in the art may be made in the arrangement, operation and details of the systems and methods disclosed herein without departing from the spirit and scope as defined in any appended claims.

Claims

1. 1. A computer-implemented method for creating an immersive audio experience, the method comprising: receiving a user selection of an audio track from a user computing device via a user interface; retrieving the audio track from an audio source in response to receiving the user selection; analyzing the audio track to determine one or more audio track characteristics; generating music analysis data based on the one or more audio track characteristics; generating vibes data based on the music analysis data, the vibes data including rules for applying one or more audiovisual (A / V) instrument functions to one or more sections of the audio track; generating visualization instructions for one or more audiovisual (A / V) instruments in communication with the user computing device based on the vibration data; generating a media file including said audio track and said visualization instructions; and transmitting the media file to the user computing device, wherein: the music analysis data includes a plurality of stems of the audio track, each stem corresponding to an instrument type within the audio track; The method, wherein the rules include binding each of the one or more audiovisual (A / V) instrument functions to one of the plurality of stems.

2. The method of claim 1, wherein the music analysis data includes a plurality of stems of the audio track, and the rules for applying the one or more audiovisual (A / V) instrument functions to the one or more sections of the audio track include assigning each of the plurality of stems to a unique audiovisual (A / V) instrument among the one or more audiovisual (A / V) instruments.

3. The method of claim 1 , wherein the one or more track characteristics include at least one of a genre, a time signature, beats per minute (BPM), a stem, an importance, a frequency, or a phrase of the audio track.

4. The method of claim 1, wherein the music analysis data includes a plurality of stems of the audio track, each stem corresponding to an instrument type within the audio track, and the rules for applying the audiovisual (A / V) instrument functions to the one or more sections of the audio track include assigning each instrument type to a unique audiovisual (A / V) instrument among the one or more audiovisual (A / V) instruments.

5. The method of claim 1, further comprising receiving via the user interface a virtual map representing locations of the one or more audiovisual (A / V) instruments in the user's media space, wherein the rules for applying the one or more audiovisual (A / V) instrument functions to one or more sections of the audio track are further based on the locations of the one or more audiovisual (A / V) instruments.

6. The method of claim 1, comprising identifying an audiovisual (A / V) equipment type for each of the one or more audiovisual (A / V) instruments communicating with the user computing device, and determining an audiovisual (A / V) equipment function for the one or more A / V instruments based on the respective audiovisual (A / V) equipment type.

7. 1. A system for creating an immersive audio experience, the system comprising: a user computing device configured to wirelessly communicate with a plurality of audiovisual (A / V) devices, the user computing device including one or more processors in communication with a memory including processor-executable instructions for providing a graphical user interface for receiving a user selection of an audio track; and one or more remote computer servers in electronic communication with the user computing device, the one or more remote computer servers comprising: responsive to receiving the user selection of the audio track, obtaining the audio track from an audio source; analyzing the audio track to determine one or more time-encoded sections of the audio track; identifying a plurality of stems of the audio track each corresponding to an element in the audio track, a first stem of the plurality of stems corresponding to a first element and a second stem of the plurality of stems corresponding to a second element; assigning, for each time-coded section of the audio track, the first stem to a first audiovisual (A / V) device of the plurality of A / V devices based on the first factor; assigning the second stem to a second audiovisual (A / V) device of the plurality of audiovisual (A / V) devices based on the second factor; and generating first visualization instructions for the first audiovisual (A / V) device based on audio characteristics of the first stem and second visualization instructions for the second audiovisual (A / V) device based on audio characteristics of the second stem; generating a media file including the first visualization instructions, the second visualization instructions, and the audio track; and transmitting the media file to the user computing device, wherein: The system, wherein the memory of the user computing device further includes processor-executable instructions for sending the first visualization instructions to the first audio-visual (A / V) device, sending the second visualization instructions to the second audio-visual (A / V) device, and dynamically adjusting the first and second audio-visual (A / V) devices during each of the one or more time-coded sections of the audio track.

8. The system of claim 7, wherein the one or more memories further include processor-executable instructions for determining a genre of the audio track, and generating the first and second visualization instructions is further based on the genre.

9. The system of claim 7, wherein the audio characteristics of the first and second stems include at least one of genre, time signature, beats per minute (BPM), frequency, or phrase.

10. 8. The system of claim 7, wherein analyzing the audio track comprises using one or more artificial intelligence (AI) engines or machine learning techniques.

11. The system of claim 7, wherein the elements are at least one of vocals, drums, instruments, and other musical elements.

12. The system of claim 7, wherein the media file and the audio track are transmitted to the user computing device using a streaming protocol.

13. The system of claim 12 , wherein the streaming protocol includes binding portions of the media file to associated portions of the audio track.

14. 1. A computer-implemented method for creating an immersive audio experience, the method comprising: receiving a user selection of an audio track from a user computing device via a user interface; analyzing the audio track to determine one or more time-encoded sections of the audio track; identifying one or more stems of the audio track, each of the one or more stems corresponding to one or more instrument types of the audio track; assigning one or more audiovisual (A / V) instrument types to each of the one or more stems based on the one or more instrument types of each stem; and analyzing each of the one or more stems to determine one or more audio characteristics of each of the one or more stems. assigning, for each time-coded section of the audio track based on the one or more audio characteristics, audiovisual (A / V) instrument functions associated with each of the one or more A / V instrument types to each of the one or more stems; generating vibration data including bindings between each of the assigned audiovisual (A / V) instrument functions and each of the one or more stems; generating a media file including said audio track and said vibration data; transmitting the media file to the user computing device, wherein: The method of claim 1, wherein the user computing device is configured to send visualization instructions to one or more audiovisual (A / V) instruments in communication with the user computing device based on the vibration data so as to dynamically adjust the one or more A / V instruments during each of the one or more time-coded sections of the audio track.

15. The method of claim 14, further comprising determining a genre of the audio track, and wherein the assignment of audiovisual (A / V) instrument functions is further based on the genre.

16. The method of claim 14 , wherein the one or more audio characteristics include at least one of genre, meter, beats per minute (BPM), frequency, or phrase.

17. The method of claim 14 , wherein the one or more instrument types are at least one of vocals, drums, instruments, and other musical elements.

18. The method of claim 14, further comprising identifying one or more user audiovisual (A / V) instruments in communication with the user computing device, and assigning each of the one or more user audiovisual (A / V) instruments to each of the one or more systems.

19. 15. The method of claim 14, wherein the media file is transmitted to the user computing device using a streaming protocol, the streaming protocol including binding the vibe data to the one or more sections of the audio track.

Images