User interface for hierarchical musical composition

Abstract

The present disclosure provides systems, devices, and methods for using a user interface associated with musical composition. An example method may comprise outputting interface objects representing musical properties and establishing hierarchical relationships between the objects. A first interface object may visually represent a first musical property. A hierarchical relationship may be determined between the first object and a second object, indicating inheritance of the first property. Changes to the first property in the first object may update the second object.

Description

USER INTERFACE FOR HIERARCHICAL MUSICAL COMPOSITION CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims priority to and the benefit of U.S. Provisional Patent Application No.63 / 624,653, filed January 24, 2024, which is hereby incorporated by reference in its entirety. BACKGROUND

[0002] Digital audio workstations (DAWs) and other similar interfaces have become important tools for creating and editing music. These software applications typically provide a multi-track interface where users can record, edit, and mix audio and MIDI data. While DAWs allow users to indicate individual notes, it is challenging for users to visualize and manipulate higher-level musical structures like scales, chords, and harmonic progressions. Users have to manually adjust individual notes, which can be time-consuming for complex harmonic structures, and these tools lack the ability to easily visualize and manipulate the overall harmonic structure of a composition. Thus, there is a need for tools that can bridge the gap between low-level musical data and higher-level musical concepts such that users can easily manipulate musical information according to higher-level musical concepts. SUMMARY

[0003] This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the detailed description. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.

[0004] Methods and systems are described for hierarchical organization of musical scales and transposition (e.g., vectoral transposition) of musical objects in a user interface. An example user interface may output interface objects representing musical properties in a composition, with hierarchical relationships between objects indicating inheritance of properties. Scale objects may define musical scales applied to descendant objects, while pattern objects may indicate sequences of musical notes associated with instruments, with notes potentially defined relative to scales. The user interface may allow editing of scales, patterns, and hierarchical relationships. Pose objects can apply operations like transposition or voice leading to modify objects for specified time ranges. The user interface may include timeline and tree views for visualizing and manipulating the hierarchical structure.Transposition operations can include chromatic, scalar, and chordal modifications. The user interface may also process audio files or composition representations to generate hierarchical relationship data for output via a user interface. The user interface may cause output the musical composition as stored data or an audio performance via a speaker.

[0005] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to limitations that solve any or all disadvantages noted in any part of this disclosure.

[0006] Additional advantages will be set forth in part in the description which follows or may be learned by practice. It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive. BRIEF DESCRIPTION OF FIGURES

[0007] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments and together with the description, serve to explain the principles of the methods and systems.

[0008] FIG.1 shows a block diagram of an example system for managing a musical composition.

[0009] FIG.2 shows an example method for managing a musical composition.

[0010] FIG.3 shows an example timeline of a project created in a popular DAW.

[0011] FIG.4 shows an example Grid view of the timeline in an example user interface.

[0012] FIG.5 shows an example Tree view of the timeline in an example user interface.

[0013] FIG.6 shows an example command to create multiple interface objects in an example user interface.

[0014] FIG.7 shows the result of executing the command from FIG.6 in an example user interface.

[0015] FIG.8 shows an example Pattern Clip in an example timeline of a user interface.

[0016] FIG.9 shows an example dropdown editor that opens when a Pattern Clip is double-clicked.24908-0584-6290.1

[0017] FIG.10 shows an example dedicated editor of a Pattern containing separate tabs for features and a list of presets.

[0018] FIG.11 shows an example Scale Clip transposing a C minor scale to an E minor scale in the timeline.

[0019] FIG.12 shows an example Pose Clip that targets a Pattern Track and its Pattern Clip.

[0020] FIG.13 shows an example transposition section of the dropdown editor of a Pose Clip.

[0021] FIG.14 shows an example voice leading section of the dropdown editor of a Pose Clip.

[0022] FIG.15 shows an example miscellaneous transformation section of the dropdown editor of a Pose Clip.

[0023] FIG.16 shows an example tree created with the VST plugin in Tree View.

[0024] FIG.17 shows an example of how the non-real-time plugin transposes highlighted notes within a notation program.

[0025] FIG.18 shows an example Tymoczko's model of the quadruple hierarchy.

[0026] FIG.19 shows an example C major chord written in staff notation being transposed along different scales.

[0027] FIG.20 shows an example C major chord written in piano roll notation being transposed along different scales.

[0028] FIG.21 shows an example Pattern Track nested within a Scale Track.

[0029] FIG.22 shows an example C major chord being transposed along different scales.

[0030] FIG.23 shows an example main four-bar motif of Scriabinism in F major.

[0031] FIG.24 shows an example transposition of the music shown in FIG.23.

[0032] FIG.25 shows an example user interface showing the transposed motif from FIG. 24.

[0033] FIG.26 shows an example musical composition in which the right hand of the motif is transposed.

[0034] FIG.27 shows an example user interface showing the first half of the transposed motif from FIG.26.

[0035] FIG.28 shows an example Timeline, modelled on the DAW.

[0036] FIG.29 shows an example Pattern Editor.

[0037] FIG.30 shows an example Scale Editor.34908-0584-6290.1

[0038] FIG.31 shows an example Instrument Editor.

[0039] FIG.32 shows an example Toolbar split into two rows.

[0040] FIG.33 shows an example File Control section of the Toolbar.

[0041] FIG.34 shows an example Audio Transport section of the Toolbar.

[0042] FIG.35 shows a Pattern Toolkit section of the Toolbar.

[0043] FIG.36 shows a block diagram illustrating an example computing device. DETAILED DESCRIPTION

[0044] The following description sets forth exemplary aspects of the present disclosure. It should be recognized, however, that such description is not intended as a limitation on the scope of the present disclosure. Rather, the description also encompasses combinations and modifications to those exemplary aspects described herein.

[0045] The disclosed techniques may provide a novel approach to organizing and manipulating musical compositions in user interfaces, such as digital audio workstations and notation programs. A user interface may be implemented that allows for hierarchical organization of musical scales and transposition of musical elements. This approach may enable more intuitive and flexible ways of working with musical structures compared to traditional digital audio tools. By providing new ways to conceptualize and manipulate musical structures digitally, the disclosed techniques may enhance the creative process for composers and music producers working with digital tools. The hierarchical approach may allow more natural representation of musical relationships, while vectoral transposition may enable more powerful and flexible transformations.

[0046] The disclosed methods and systems present significant technical improvements over existing digital audio workstations and notation programs, specifically in the realm of user interfaces. Traditional digital music tools often rely on linear and rigid structures that can limit a user’s creative process. In contrast, the hierarchical organization and transposition techniques disclosed herein provide a more flexible and intuitive approach, which constitutes a clear advancement in the state of the art.

[0047] One of the innovations of the present disclosure includes the ability to hierarchically organize interface objects associated with corresponding musical properties (e.g., musical scales and elements) within the user interface. This hierarchical structure allows users to visualize and manage complex musical relationships more naturally, aligning more closely with how musicians conceptualize composition and arrangement. By facilitating44908-0584-6290.1an organized and layered view of musical elements, users can navigate and manipulate their compositions with greater ease and precision. This represents a substantial improvement over conventional flat or minimally hierarchical interfaces, which do not effectively manage intricate musical data structures. This also allows for complex modifications of musical structures via updating cascading changes via hierarchical relationships.

[0048] Additionally, the implementation of vectoral transposition as a scheduled event within the user interface introduces a novel mechanism for transforming musical elements. Unlike traditional transposition methods that may be static or simplistic, vectoral transposition allows for dynamic and multifaceted transformations. Users can apply scalar offsets at various hierarchical levels, enabling complex and musically meaningful modifications to compositions. This capability not only enhances the creative possibilities, for instance by allowing musically meaningful edits to completed passages of music, but also streamlines the workflow, reducing the cognitive load on the user by automating intricate transpositional processes. Moreover, the disclosed user interface supports custom transposition operations defined by users, further advancing the technical field. By allowing users to create and apply bespoke transpositions, the interface adapts to diverse compositional needs and preferences, making it a versatile tool for a wide range of musical styles and methodologies. This customization feature highlights the interface’s adaptability and user- centric design.

[0049] In summary, the disclosed user interface methods and systems provide technical improvements that enhance the functionality, usability, and creative potential of digital music tools. These advancements address existing limitations and offer a more powerful and flexible environment for composers and music producers.

[0050] FIG.1 shows a block diagram of an example system 100 for managing a musical composition. The system 100 may comprise a storage device 102 (e.g., server based storage), a network device 104 (e.g., gateway device, access point, or combination thereof), a user device 106, an application device 108 (e.g., an application device, application server), an output device 110, or a combination thereof. It should be noted that while the singular term device is used herein, it is contemplated that some devices may be implemented as a single device or a plurality of devices (e.g., via load balancing). The storage device 102, the network device 104, the user device 106, the application device 108, and / or the output device may each be implemented as one or more computing devices. Any device disclosed herein may be54908-0584-6290.1implemented using one or more computing nodes, such as virtual machines, executed on a single device and / or multiple devices.

[0051] The storage device 102, the network device 104, the user device 106, the application device 108, and / or the output device 110 may be communicatively coupled via one or more networks, such as a first network 112 (e.g., a wide area network) and a second network 114 (e.g., one or more local area networks). The first network 112 and / or second network 114 may comprise wired links, wireless links, a combination thereof, and / or the like. The first network 112 and / or second network 114 may comprise routers, switches, nodes, gateways, servers, modems, and / or the like.

[0052] The application device 108 may be configured to provide services for one or more applications on the user device 106. For example, the application device 108 may generate application data associated with the one or more application services. The application data may comprise data for a user interface, data to update a user interface, data for an application session associated with the user device 106, and / or the like. The application data may comprise data associated with management of a musical composition. The application data may comprise audio clips, note data, pitch data, scale data, relationship data (e.g., relationships between user interface objects) user preference data, and / or the like.

[0053] The output device 110 may comprise a speaker, such as a smart speaker (e.g., a device comprising a speaker, a computer processor, and a microphone). The output device 110 may be configured to receive audio data, musical composition data, textual data, graphic data, and / or a combination thereof. For example, if the output device is a speaker, the output device 110 may be caused by another device in the system 100 to play an audio file (e.g., a performance of a musical composition. The output device 110 may comprise any other output device, such as a printer, a display, an instrument (e.g., automated instrument player component), and / or the like. If the output device is a speaker, the output device 110 may be caused by another device in the system 100 to print text and / or graphic representing a musical composition.

[0054] The network device 104 may be comprise a computing device, a gateway device, an access point (e.g., wireless access point), a router, a modem, device controller (e.g., automation controller, security controller, premises health controller, content device controller) a combination thereof, and / or the like. The network device 104 may be configured to communicate using the one or more second networks 114 at a user premises. The network device 104 may be configured to facilitate communication between the user device 106 and64908-0584-6290.1the output device 110. The network device 104 may be configured to facilitate communication between the user device 106, the storage device 102 and the application device 108.

[0055] The user device 106 may comprise a computing device, a smart device (e.g., smart glasses, smart watch, smart phone), a mobile device, a tablet, a computing station, a laptop, a digital streaming device, a set-top box, a streaming stick, a television, and / or the like. In some scenarios, a user may have multiple user devices, such as a mobile phone, tablet device, a smart watch, smart glasses, virtual reality headset, mixed reality headset, a combination thereof, and / or the like. The user device 106 may be configured to communicate with the storage device 102, the network device 104, the application device 108, the output device 110, and / or the like. The user device 106 may be configured to output a user interface 116. For purposes of illustration, a simplistic representation is shown in FIG.1, but example more detailed representations are provided further herein and shown throughout the figures. The user interface 116 may be output via the user interface via an application, service, and / or the like, such as a web browser, executable application, and / or the like. The user interface 116 may receive application data from the application device 108. The application data may be processed by the user device 106 to cause display of the user interface.

[0056] The user interface 116 may be displayed on a display of the user device 106. The display may comprise a television, screen, monitor, projector, and / or the like. The user interface 116 may comprise a music composition application, a notation application, a digital audio workstation, and / or the like. The user interface 116 may be configured to allow users to notate a pattern of notes (e.g., or pitches), manipulate the notes, store the notes, and / or the like. The application may be configured to allow control of and / or sending commands to the output device 110.

[0057] The user interface 116 may be configured to implement the method of FIG.2. In some aspects, the user interface 116 displayed on the user device 106 may execute the steps outlined in the example method for managing a musical composition. The user interface 116 may allow users to interact with and manipulate musical objects in ways that correspond to the processes described in FIG.2. For example, the user interface 116 may facilitate outputting interface objects representing musical properties, determining hierarchical relationships between objects based on user input, displaying representations of those relationships, and / or the like. The interface may also enable users to create and modify scale objects, pattern objects, and / or pose objects as part of implementing the method shown in74908-0584-6290.1FIG.2. By integrating these functionalities, the user interface 116 may provide a comprehensive environment for users to engage with the hierarchical organization and manipulation of musical elements as outlined in the example method.

[0058] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may cause output of an interface window 118. The interface window 118 may comprise one or more of a plurality of interface objects for representing a musical composition. The user interface 116 may allow users to define and / or associate musical properties with interface objects. For example, the user interface 116 may comprise preset patterns, scales, and / or instruments to facilitate creation of transpositions and other musical elements. These present musical elements may be selected by users and / or users may define their own musical elements to associate with interface objects.

[0059] The interface window 118 may comprise a timeline 120. User interface objects (e.g., or associated musical properties, such as patterns or applied transpositions) may be overlayed on the timeline 120. For example, on the left of the timeline (e.g., either separately or above the timeline) is shown an interface area with different interface objects (e.g., an interface box) indicating a type of interface object, interface object name, buttons for editing object properties, and / or the like (e.g., as shown in more detail in FIG.8). Location on the timeline 120 may represent scheduling of different interface objects having various musical properties. For example, the beginning of a musical composition may correspond to a left most position in the timeline 120, and the timeline 120 may progress to the right with increasing time intervals that may be associated with interface objects. This timeline view may allow users to visualize and manipulate musical elements in a chronological sequence. Users may be able to add, remove, and rearrange interface objects representing different musical properties within this timeline.

[0060] In some scenarios, the timeline 120 may not be present, but other representations may be used, such as a tree node structure. Placement in the interface window 118 of interface objects may represent and / or define hierarchical relationships between interface objects. The representations on the user interface 116 may correspond to relationships that are stored in data structures, such as a database, markup language, and / or the like that allows for the definition of musical properties, the hierarchical relationships between interface objects (e.g., or the properties they represent), and / or the like.

[0061] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may cause output of a first interface object 122 for visually representing84908-0584-6290.1a first musical property in a musical composition. The user interface 116 may cause output of a second interface object 124 for visually representing a second musical property in the musical composition.

[0062] The first interface object 122 may be a different type of interface object than the second interface object. The first interface object 122 may comprise a scale object indicating a musical scale (e.g., a musical scale may be any collection of notes) to apply to one or more descendent objects via inheritance from the first interface object. The first musical property may comprise an indication of one of a plurality of available musical scales. The second interface object 124 may be associated with a second musical property different from the first musical property associated with the first interface object 122. The second interface object 124 may comprise a pattern object indicating (e.g., or associated with) a pattern 126, such as a musical pattern, or sequence of musical notes. The pattern object may indicate an instrument associated with the sequence of musical notes. The second interface object 124 may allow a user to edit the sequence of musical notes. The second interface object 124 may comprise a midi track.

[0063] As previously explained, the user interface 116 may comprise a timeline view showing the one or more interface objects along (e.g., or associated with a timeline 120). For example, in the timeline view the interface objects (e.g., the first interface object 122, the second interface object 124) may be arranged as tracks, where each object has its own track (e.g., and tracks may be like rows vertically stacked above / below each other). In the timeline view, hierarchical relationships may be representation by indentation 125 from the left side offsetting one object from another.

[0064] In addition or in the alternative, the user interface 116 may comprise a tree view that represents the one or more interface objects as a hierarchical tree graph (e.g., as shown in FIG.5). For example, a respective node may be shown for each corresponding interface object and a line between two nodes of the respective nodes represents a hierarchical relationship between the two nodes. The tree view may provide users with a clear visualization of the nested structure of the musical composition.

[0065] The user interface 116 may be configured to allow a user to edit the hierarchical relationship. The user interface 116 may be configured to allow a user to modify multiple objects in a hierarchy of interface objects to define musical properties inherited by multiple descendants of an interface object. The user interface 116 may comprise one or more of a digital audio workstation or a notation program.94908-0584-6290.1

[0066] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may determine a hierarchical relationship between the first interface object 122 and a second interface object 124. The hierarchical relationship between the first interface object 122 and a second interface object 124 may be determined based on user input via the user interface. The hierarchical relationship may indicate inheritance of the first musical property from the first interface object 122 to the second interface object 124 such that a change made to the first musical property in the first interface object 122 causes an update to the second interface object 124 (e.g., to the pattern 126). For example, changes in the user interface 116 may cause cascading changes based on hierarchical relationships of user interface objects. Updating the first musical property may cause a plurality of updates to a plurality of interface objects inheriting via corresponding hierarchical relationships from the first interface object 122.

[0067] The user input may comprise a user indicating a type of object (e.g., scale object, pattern object, etc.) of the second interface object 124. The user input may indicate a relationship between the first interface object 122 and the second interface object 124. Determining the hierarchical relationship may comprise processing one or more of an audio file or a representation of the musical composition and generating, based on the processing, data indicative of a hierarchical relationship. Processing may comprise determining a scale object indicating a musical scale, determining a pattern object indicating a series of musical notes, and determining the hierarchical relationship between the scale object and the pattern object.

[0068] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may be configured to cause output of a representation of the hierarchical relationship. The output of a representation of the hierarchical relationship may be caused via the user interface 116. Causing output may comprise one or more of updating the user interface 116, causing storage of the hierarchical relationship (e.g., at the storage device 102, at the user device 106, and / or at the application device 108), or sending via a network data indicative of the hierarchical relationship to a computing device outputting the user interface (e.g., the output device 110).

[0069] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may be configured to determine based on user input, a pose object 128 (e.g., transposition object, transformation object, modification object) configured to apply an operation that modifies the second interface object 124 (e.g., or its musical property, such as104908-0584-6290.1a portion of the pattern 126) for a time range. The operation may comprise one or more of transposition of notes, changing a scale, modifying a note, modifying musical properties of descendent interface objects, generating a voice leading, changing a set of musical notes to another set of musical notes, or changing a set of pitches to another set of pitches. The user interface 116 may be configured to allow a user to define a transposition operation that comprises one or more of a chromatic modification, a scalar modification, or a chordal modification.

[0070] The user interface 116 (e.g., or the user device 106, the application device 108, a combination thereof) may be configured to cause output of the musical composition by one or more of storing data indicative of the musical composition or causing a speaker to output an audio performance of the musical composition.

[0071] The user interface 116 may be configured to implement text and / or character based transposition commands, such as a vectoral transposition (e.g., as shown in FIGs.6-7). Vectoral transposition may be implemented as a scheduled event that can be applied to an interface object in the user interface. In some cases, a vectoral transposition may comprise a record of scalar offsets that are applied to affected notes. The system may iteratively apply current transpositions from the highest ancestor track down to the current track every tick to produce transformed musical objects.

[0072] Transpositions may be labeled using a format that indicates different types of offsets. For example, a transposition may be labeled as Nx • T(X1, X..., Xn) • tx, where: Nx may represent the offset along the chromatic scale, T(X1, X..., Xn) may represent the offsets along parent scales, and tx may represent the offset along the object’s intrinsic scale, defined as the octave-repeating scale containing the object’s notes.

[0073] This labeling system may allow for precise specification of complex transpositions involving multiple scales.

[0074] When applied to a Scale Track (e.g., the first interface object 122), a transposition (e.g., as a transposition command and / or a pose object) may modify the scale itself. When applied to a Pattern Track, a transposition may modify the patterns of notes within that track. The user interface 116 may automatically update all descendant objects to reflect transpositions applied to parent tracks.

[0075] The user interface 116 may be configured to allow users to define custom musical transformations. These transformations may comprise one or more of: A chromatic114908-0584-6290.1modification (e.g., shifting notes by a fixed number of semitones), a scalar modification (e.g., moving notes along a specified scale), or a chordal modification (e.g., rotating notes within a chord or scale).

[0076] Users may be able to schedule transpositions to occur at specific time points within tracks (e.g., or interface object). The user interface 116 may provide keyboard shortcuts to efficiently create and edit transpositions. For example, holding certain keys may target specific scales, while pressing number keys may create transpositions with corresponding offsets.

[0077]

[0078] The user interface 116 may be configured to allow a user to edit the hierarchical relationship between interface objects. Users may be able to drag and drop nodes in the tree view to restructure the hierarchy, or use dedicated controls to modify parent-child relationships between musical elements.

[0079] The user interface 116 may comprise a plurality of additional editors that allow for a user to adjust various musical elements. A Scale Editor may be included for creating and editing musical scales. The Scale Editor may provide features such as adding or removing notes from a scale, transposing the entire scale, and inverting the scale. Users may be able to preview the sound of their custom scales and save them for future use.

[0080] A Pattern Editor may be included for creating and editing musical patterns. The Pattern Editor may offer features such as adding or removing notes, transposing patterns, inverting patterns, contracting or expanding note durations, and even randomizing patterns to generate new ideas. Users may be able to edit patterns using both a piano roll interface and traditional music notation.

[0081] An Instrument Editor may be included for customizing instrument sounds. The Instrument Editor may provide various audio effects such as warp, reverb, chorus, and delay. Users may be able to adjust parameters for each effect to shape the timbre of their instruments.

[0082] In some cases, the user interface 116 may be implemented as a VST (Virtual Studio Technology) plugin for use in existing DAWs (Digital Audio Workstations) and notation programs. This may allow users to integrate the hierarchical and vectoral composition tools into their existing workflows and software environments.

[0083] The user interface 116 may include a non-real-time plugin mode for use in DAWs and notation programs to apply transformations to selected notes. In this mode, users may be124908-0584-6290.1able to highlight a section of their composition, open the plugin interface, and apply complex transformations based on the hierarchical scale structures and vectoral transpositions.

[0084] By providing these various interface elements and editing tools, users may be able to create and manipulate complex musical structures in an intuitive and visually informative way. The combination of timeline and hierarchical views, along with specialized editors for scales, patterns, and instruments, may offer a comprehensive environment for exploring and developing musical ideas. The hierarchical organization of scales, transposition operations (e.g., via interface objects of vectoral commands), and user interface elements may work together to enable advanced musical composition in a digital audio workstation or notation program. A device comprising one or more processors and a memory storing instructions may implement this system.

[0085] FIG.2 shows an example method 200 for managing a musical composition. The method 200 may allow of creating and manipulating musical structures via the user interface 116 of FIG.1. The method 200 may comprise a computer implemented method for providing a service (e.g., user interface service, a musical compositing editor service, an audio editing service). A system and / or computing environment, such as the system 100 of FIG.1 and / or the computing environment of FIG.36, may be configured to perform the method 200. The method 200 may be performed in connection with the system illustrated in FIG.1. Any step or combination of steps of the method 200 may be performed by a computing device, network device, storage device, server device, and / or user device, such as any of the devices shown in FIG.1. The method 200 may be implemented as software instructions stored on a non-transitory computer-readable medium and executed by one or more processors. In some aspects, the method 200 may be performed by a cloud-based service accessible via a network. The method 200 may interface with other software applications or services, such as digital audio workstations, notation programs, or music production tools. Steps of the method 200 may be performed in parallel or in a different order than shown in FIG.2. Additional steps not shown may be included, and some steps may be omitted, combined, or broken into sub- steps.

[0086] At step 202, a user interface may be caused to output a first interface object for visually representing a first musical property in a musical composition. The user interface may output a second interface object visually representing a second musical property in the musical composition. The first interface object may be a different type of interface object than a second interface object. The first interface object may comprise a scale object134908-0584-6290.1indicating a musical scale to apply to one or more descendent objects via inheritance from the first interface object. The first musical property may comprise an indication of one of a plurality of available musical scales. The first interface object may allow a user to edit the musical scale independent from musical notes associated with the second interface object. The user interface may comprise a timeline view showing the one or more interface objects along a timeline. The user interface may comprise a tree view that represents the one or more interface objects as a hierarchical tree graph. A respective node may be shown for each corresponding interface object and a line between two nodes of the respective nodes represents a hierarchical relationship between the two nodes. The user interface may be configured to allow a user to edit the hierarchical relationship. The user interface may be configured to allow a user to modify multiple objects in a hierarchy of interface objects to define musical properties inherited by multiple descendants of an interface object. The user interface may comprise one or more of a digital audio workstation or a notation program. The user interface may provide drag-and-drop functionality to allow users to easily rearrange the hierarchical relationships between interface objects. The first interface object may include controls for adjusting properties like key signature, mode, or custom scale definitions. The user interface may provide different viewing modes, such as a compact view that collapses nested objects or an expanded view showing full hierarchical relationships. The interface objects may be zoomable to show different levels of detail. The user interface may include a search or filtering function to quickly locate specific interface objects within a complex hierarchical structure.

[0087] At step 204, a hierarchical relationship between the first interface object and a second interface object may be determined. The hierarchical relationship between the first interface object and a second interface object may be determined based on user input via the user interface. The hierarchical relationship indicates inheritance of the first musical property from the first interface object to the second interface object such that a change made to the first musical property in the first interface object causes an update to the second interface object. For example, updating the first musical property may cause a plurality of updates to a plurality of interface objects inheriting via corresponding hierarchical relationships from the first interface object. The hierarchical relationship may be visually represented in the user interface, such as through indentation, connecting lines, or nested containers. The system may automatically suggest potential hierarchical relationships based on musical analysis of the composition. Users may be able to define custom inheritance rules or override default144908-0584-6290.1inheritance behavior for specific objects. The hierarchical relationships may support multiple inheritance, allowing a single object to inherit properties from multiple parent objects. The system may provide visual feedback when changes to parent objects affect child objects, such as highlighting updated elements. The hierarchical structure may allow for the creation of reusable musical components that can be easily applied across different parts of a composition.

[0088] The second interface object may be associated with a second musical property different from the first musical property associated with the first interface object. The second interface object may comprise a pattern object indicating a sequence of musical notes. The pattern object may indicate an instrument associated with the sequence of musical notes. The second interface object may allow a user to edit the sequence of musical notes (e.g., and / or the musical instrument). The user input may comprise a user indicating a type of object of the second interface object and a relationship between the first interface object and the second interface object. The second interface object may comprise a midi track. Determining the hierarchical relationship may comprise processing one or more of an audio file or a representation of the musical composition and generating, based on the processing, data indicative of a hierarchical relationship. Processing may comprise determining a scale object indicating a musical scale, determining a pattern object indicating a series of musical notes, and determining the hierarchical relationship between the scale object and the pattern object. The second interface object may include controls for adjusting properties like articulation, dynamics, or effects specific to the associated instrument. The pattern object may support different note input methods, such as step sequencing, real-time recording, or algorithmic generation. The system may provide intelligent quantization options to align recorded notes with the inherited scale or rhythm properties. The second interface object may allow for the application of MIDI effects and / or transformations that account for the inherited musical properties from parent objects. The pattern object may include visualization options to display the notes in different formats, such as traditional notation, piano roll, or tablature, depending on the instrument type.

[0089] At step 206, output of a representation of the hierarchical relationship may be caused. The output of a representation of the hierarchical relationship may be caused via the user interface. Causing output may comprise one or more of updating the user interface, causing storage of the hierarchical relationship, or sending via a network data indicative of the hierarchical relationship to a computing device outputting the user interface. The method154908-0584-6290.1may comprise causing output of the musical composition by one or more of storing data indicative of the musical composition or causing a speaker to output an audio performance of the musical composition. The representation of the hierarchical relationship may be exportable in various formats, such as XML or JSON, to facilitate interoperability with other music software. The system may generate a textual or graphical summary of the hierarchical structure, providing an overview of the composition's organization. The output may include an interactive visualization that allows users to explore the hierarchical relationships, such as expanding or collapsing branches of the structure. The system may provide options to compare different versions or arrangements of the hierarchical structure, helping users explore alternative compositional approaches. The output may include analytics or statistics about the hierarchical structure, such as the depth of nesting or the distribution of different object types.

[0090] The method may further comprise determining, based on user input, a pose object configured to apply an operation that modifies the second interface object for a time range . The operation may comprise one or more of transposition of notes, changing a scale, modifying a note, modifying musical properties of descendent interface objects, generating a voice leading, changing a set of musical notes to another set of musical notes, or changing a set of pitches to another set of pitches. For example, the user interface may be configured to allow a user to define a transposition operation comprising one or more of a chromatic modification, a scalar modification, or a chordal modification. The pose object may be configured to allow multi-step transformations. The transformations may be saved and reused across different parts of the composition, or from one composition to another. Users may be able to schedule pose objects to activate at specific times and / or in response to musical events, allowing for dynamic changes in the composition. The user interface may provide a library of pre-defined pose objects for common musical transformations. The user interface may allow users to customize and / or combine predefined pose objects. The pose object may include options for probabilistic and / or generative operations (e.g., to add controlled randomness into the composition). The user interface may allow for the visualization of pose object effects, such as showing before and after states and / or animating the transformation process.

[0091] Additional Discussion

[0092] Disclosed herein are a system and a method for hierarchical organization of musical scales (e.g., any collection of notes, and encompassing chords or subchordal164908-0584-6290.1collections), vectoral transposition of musical objects (e.g., voicings and motives), and spatiotemporal portaling of track media in a digital audio workstation.

[0093] By way of background, musical notation, in its current form, is not entirely optimized for the needs of the composer. Even though it is common for composers to think in terms of musical structures like chords and scales, current tools for notating music do not provide a means for creating or manipulating or visualizing these structures. Instead, composers must rely on their own abstractions to convert a compactified stream of pitches into a collection of intricately woven musical structures, which introduces confusion and tedium into every user’s workflow.

[0094] The disclosed approach provides a novel form of music notation by upgrading the interface of a digital audio workstation (DAW) or notation program with the means to design and visualize musical structures of arbitrary size and complexity and the ability to create, organize, and manipulate nested musical structures. By allowing composers to unravel, notate, and reason about their musical structures, they are granted the ability to organize their harmonies, navigate between different tonalities, and automatically perform common operations like transposition, modulation, and efficient voice leading. With this innovative functionality, users can be empowered to rapidly explore and develop new ideas without obscuring any of their musical information along the way, alleviating countless hours of unnecessary and nebulous labor.

[0095] The disclosed approach can be used in tandem with or in place of any digital audio workstation for purposes ranging from musical composition and live performance to theoretical analysis and educational instruction. In the future, the disclosed approach may be realized within novel software, integrated within the commercial plugin ecosystem, and / or licensed to digital music software companies.

[0096] Hierarchical Organization of Musical Scales:

[0097] Digital audio workstations (DAWs) typically allow users to work with tracks that store musical notes as numbers from 1 to 127 in accordance with the Musical Instrument Digital Interface (MIDI) standard. Despite the fact that musicians tend to think in terms of nestable structures, like chords that move within scales that move within a chromatic collection, current DAWs do not offer the means to design or visualize musical structures beyond a single layer of MIDI notes. This means that composers must rely on their own abstractions to convert a compactified sequence of pitches into a collection of intricately174908-0584-6290.1woven musical structures, which introduces inefficient and potentially inaccurate labor into every user’s workflow.

[0098] By upgrading digital audio workstations with the ability to nest scale-containing tracks within one another, it becomes possible for musicians to design recursively nested musical scales and dynamically bind MIDI notes to any matching scale degree. With this unprecedented degree of expression, a composer can explicitly customize the architecture of their music and automatically perform complex calculations that are required for common operations like scalar transposition and voice leading.

[0099] Vectoral Transposition of Musical Objects:

[0100] Since DAWs do not provide hierarchically organized musical structures, a user can only transpose a collection of MIDI notes along a one-dimensional scale. However, there is frequently a need to transpose individual notes along nested scales or to apply an aggregate transposition comprising (e.g., or consisting) of several independent movements, for which a composer will have to both manually perform the chain of operations and also keep track of every relevant scale as the music is transformed. This creates an unnecessary cognitive strain on the composer and makes it difficult to conceptualize or perform complex transpositions.

[0101] By defining a transposition as a schedulable, vector-based operation that applies a series of offsets to a note (e.g., moving it along multiple collections independently and simultaneously) it becomes possible to precisely model and instantly calculate musical motion along infinitely adjustable axes without having to rewrite or overwrite any individual notes. With this flexible and retractable approach to transposition, composers can easily develop their music and painlessly revise entire compositions to explore new harmonic territory in a matter of minutes, an unachievable task in current DAWs.

[0102] Spatiotemporal Portaling of Track Media:

[0103] Although it is common to send an audio effect or signal from one track to another, there is no existing mechanism for rerouting musical notes or structures. Typically, composers must slice their music into fragments in order to transport notes across tracks or timepoints, moving those slices manually, which may render it difficult to relate different sections together after some time has passed. Additionally, should a composer wish to rejoin these sliced fragments later on, they will have to tediously find, rearrange, and merge every clip. Once a slice has been moved to multiple tracks its different manifestations typically need to be edited independently.184908-0584-6290.1

[0104] By introducing spatiotemporal portals that can directly tunnel media from one track / tick to another, it becomes possible for composers to create an explicit connection between musical fragments without obscuring their original reference. By placing an entrance portal at one coordinate and an exit portal at another coordinate, composers can transparently portal their music between different scales and instruments without having to slice out individual notes or motifs. This opens the door for a novel framework of portal-based composition with the possibility to implement algorithmic or probabilistic events in the future.

[0105] With respect to implementing the disclosed approach, a hierarchical organization of musical scales can be implemented by allowing tracks to contain a reference to a parent track and a scalar collection of notes. In a prototype, nestable Scale Tracks are implemented that correspond to musical scales and instrumental Pattern Tracks that serialize scale-based patterns into MIDI notes. Each Scale Track contains a dedicated editor for customizing its corresponding scale, which is defined using scale degrees relative to the parent track’s scale or the chromatic scale if at the highest level.

[0106] A vectoral transposition may be a scheduled event that can be placed in any track with a record of scalar offsets that are to be applied to every affected note. Every tick, each track will iteratively apply every current transposition from its highest ancestor to itself and produce a transformed musical object—a transposed scale within Scale Tracks or a transposed pattern note within Pattern Tracks. In the prototype, transpositions are labeled as Nx• T(X1, X…, Xn) • txwhere Nxis the offset along the chromatic scale, T(X1, X…, Xn) are the offsets along the parent scales, and txis the offset along the intrinsic scale of the object (i.e. producing the modes or inversions of a collection).

[0107] A spatiotemporal portal comprises (e.g., or consists) of two distinct track / tick coordinates signifying an entrance and an exit. After every media element is pre-processed (corresponding to pattern clips and transpositions in the disclosed approach), it will be rendered through any overlapping portals and chunked into fragments accordingly.

[0108] A possible limitation of the disclosed approach comes from the novelty of vectoral transpositions. Even though vectors can precisely notate transpositions, they are not in mainstream use for composition and higher dimensional musical structures are inherently difficult to reason about. Hence, there is a learning curve involved with understanding how to map transpositions to vectors and vice versa. However, composers will learn and improve with continual practice, as is required of them for all other musical skills.194908-0584-6290.1

[0109] It is envisioned that the disclosed approach can be implemented both as a commercial plugin (e.g., for both DAWs and notation programs) and as standalone commercial software that can be used for musical notation, composition, performance, and education. Additionally, since the disclosed approach directly upgrades the workflow of DAW-based composers with inventive, impactful, and time-saving features, it is believed that there will be incentive for companies (such as Avid, Apple, and Ableton) to license this technology and implement these features directly into their existing digital audio workstations and other products.

[0110] First Example Experimental Implementation of User Interface

[0111] It should be understood that the following examples of an experimental version of a user interface are provided for purposes of illustration and the disclosure is not limited to these features. Any single feature or combination of the features of this example can be combined with any of the other examples or features provided elsewhere herein.

[0112] The disclosed user interfaces may comprise a digital audio workstation (DAW). For example, the user interface may be configured to perform one or more of the following: 1) create “trees” of hierarchically nested collections (e.g. chords within scales); 2) create “motifs” out of notes with functional relationships to trees (e.g. upper scalar neighbor, a term referring to a note that is one scale step above a note belonging to a chord, which typically refers to a scalar subset one level higher than the scale); and 3) create “poses” applying complex transformations to trees and motifs (e.g. voice leading).

[0113] The following sections will first disclose an example user interface (e.g., sometimes referred to as Harmonia). The user interface may be implemented via a local application, as a web application, or a combination thereof. Each section will start with the design problem in contemporary DAWs like Ableton and Reaper, then highlight the disclosed innovation and implementation with accompanying screenshots from an example prototype. After this explanation, the next paragraphs will explain the user interface, as it can be implemented in a Virtual Sound Technology (VST) real-time and non-real-time plugin that integrates the website’s functionality with existing DAWs and notation programs.

[0114] Trees

[0115] Trees - Background: The “tracks” of DAWs

[0116] In a DAW, users may start their projects by creating musical containers called “tracks” that can schedule sounds (e.g. audio and MIDI tracks) or apply effects (e.g. automation and send tracks). The interface of the DAW is centered around a grid of cells204908-0584-6290.1called the “timeline” (e.g., FIG.3), letting users arrange tracks (the rows) and schedule musical “clips” within them (the columns). This design is conducive to musicians who record instruments or sample audio, but it does not help composers who implicitly manipulate a hierarchy of voices, chords, and scales.

[0117] FIG.3 shows an example timeline of a project created in an example DAW.

[0118] Trees - Approach: The “trees” of the disclosed user interface

[0119] In the disclosed user interface, users can create hierarchical structures or “trees” of tracks with tiers of information that cascade from ancestors to descendants. To implement this behavior, the disclosed user interface may be configured to allow tracks to nest. If a track has any children, it can pass down its properties to them. For example, a track's scale will parent the child’s scale. A track's pose will sum with the child’s pose. An instrument will play in unison, and so on. There is no limit on the depth or the size of a tree, and users can set these properties or schedule changes to them by arranging clips — ”Scale Clips” to switch scales, “Pose Clips” to apply poses, and “Pattern Clips” to switch instruments and play audio.

[0120] Trees - Implementation of the timeline

[0121] The user interface (e.g., or timeline thereof) may be implemented using various representations, such as the following two modes: Grid view (e.g., FIG.4) and Tree view (e.g., FIG.5). The Grid view imitates the timeline of the DAW, vertically grouping tracks of the same tree and horizontally indenting each track to signal its depth in the hierarchy. The Tree view may replace the timeline with a tree graph that models hierarchical data with circular nodes and linear edges.

[0122] FIG.4 shows an example Grid view of the timeline.

[0123] FIG.5 shows an example Tree view of the timeline.

[0124] Trees - Implementation of nestable tracks

[0125] Each track (e.g., or interface object) may come with additional controls to customize its properties and nest new children. Tracks (e.g., or interface objects) with scales are called Scale Tracks. These may be colored blue with buttons to change their scales. Tracks (e.g., or interface objects) with instruments are called Pattern Tracks. These may be colored green with buttons to control their audio. Every track can specify a base transposition to be applied to itself and its children, displayed in pink text within the track. All tracks may come with a label to identify their position in their tree, comprising (e.g., or consisting) of a sequence of n letters from A-Z, where n is equal to the depth of the track, and each letter corresponds to the track’s order amongst its siblings at that level.214908-0584-6290.1

[0126] Trees - Implementation of an efficient pattern matching language to create trees

[0127] Users can create trees (e.g., or hierarchical relationship between interface objects) by clicking on buttons within their tracks to nest new children and repeating as necessary—or alternatively, by entering their prompt in “TreeJS”, an efficient pattern-matching language created for the disclosed user interface. Provided below are six basic rules of TreeJS and an example of the language in the action.

[0128] Rule 1: Scales can be specified with $ and their name, pitch class, or MIDI number. Example: “$ C major chord” or “$ (C, E, G)” or “$ (60, 64, 67)”.

[0129] Rule 2: Instruments can be specified with ~ and their name or the keyword “file” to upload audio. For example: “~ piano” orstring ensemble” or “~ file”.

[0130] Rule 3: Poses can be specified with * and a list of comma-separated effects. For example: * A1, B-2 means move all notes 1 step up Track A and 2 steps down Track B.

[0131] Rule 4: A track’s children can be specified withFor example: A => B => C means track A is the parent of B, and B is the parent of C.

[0132] Rule 5: A track’s siblings can be specified withFor example: A => B + C means that A is the parent of B, and A is the parent of C.

[0133] Rule 6: An order of operations can be specified with parentheses. For example: (A => B) + C is not equal to A => (B + C)

[0134] FIGs.6-7 shows an example command to create multiple interface objects. The example command includes the following:“( $ C major => $ C maj7 => ~ piano ) + ( $ E, G#, B => ~ guitar )”. This creates Track A with a C major scale, Track AA with a nested C major 7th chord, Track AAA with a piano inheriting scales A and AA, Track B with an E major chord, and Track BA with a guitar inheriting scale B.

[0135] Motifs

[0136] Motifs - Background: The “notes” of DAWs

[0137] In a DAW, users can compose clips of notes by specifying their pitches with a number from 1-127, in accordance with the MIDI (Musical Instrument Digital Interface) protocol. To design their patterns, users are provided with a “piano roll” interface, a compact grid with one row for each pitch and one column for each subdivision. Although this approach is helpful for precisely adjusting pitch and duration, it does not intuitively visualize or incorporate other kinds of information about the notes, like which scales they belong to or which roles they serve. The presently disclosed user interface provides improvement over conventional approaches.224908-0584-6290.1

[0138] Motifs - Approach: The “motifs” of the disclosed user interface.

[0139] In the disclosed user interface, users can compose “motifs” whose notes are represented as “scale degrees” of scales in the track’s hierarchy, or possibly as intervals of notes in the hierarchy (e.g., “one scale step above scale degree 60 in the chordal hierarchy”).

[0140] To implement this behavior, the MIDI protocol may be extended with additional properties to specify the parent and degree of each note, as well as a record of numerical offsets along any tracks’ scales.

[0141] For example, a middle C note can be represented as the first degree of a C major scale, the fifth degree of an F minor scale, the second degree of a G minor chord moved up one step along the Bb major scale, and so on, depending on the choice of the user. With these bindings, the function of each note is transparent, and the notes will be able to adapt to any changes made to their scale.

[0142] Motifs - Implementation of Pattern Clips

[0143] The user interface may be configured to allows users to create a motif by arranging a new “Pattern Clip” in the timeline (e.g., FIG.8) and double clicking on the clip to edit its notes: a dropdown interface opens (e.g., FIG.9) with a virtual keyboard for users to input their pitches, an interactive sheet music score to display their notes, a heads-up display for navigation, rhythm, and dynamics, and a pop-up editor for further customization (e.g., FIG.10). By default, the software may try to assign a fitting role to each MIDI note (chord tone, diatonic neighbor, chromatic neighbor, etc.), and the algorithm will prioritize the configuration with the most depth. However, with the pop-up editor, one is free to bind each note to a different scale, to bind the pattern to another track, or to clear the bindings entirely and fix the MIDI values as pedal points.

[0144] FIG.8 shows an example Pattern Clip which references the New Pattern object.

[0145] FIG.9 shows an example dropdown editor that opens when a Pattern Clip is double-clicked.

[0146] FIG 10. shows an example dedicated editor of a Pattern containing separate tabs for features and a list of presets.

[0147] Motifs - Scale Clips

[0148] Users may change the scale of a Pattern Clip or specific track by arranging a new “Scale Clip” (e.g., FIG.11) in the timeline. The user can double click on the clip to edit its scale, and a dialog box appears to input a new scale by name, note, or MIDI number, or to define a scale mapping.234908-0584-6290.1

[0149] FIG.11 shows an example Scale Clip transposing a C minor scale to an E minor scale in the timeline.

[0150] Poses

[0151] Poses - Background: The “transpositions” of DAWs

[0152] In a DAW, users can typically work with harmony in one of two general ways. They can either constrain the key of their music, or they can transpose the notes of their patterns. However, without the hierarchical data provided by trees (as outlined in Section 1), there is no way for users to represent multilayered transpositions or voice leadings, a central feature of Western composition (Tymoczko 2023). These operations must be manually performed for each transposition and each track, and their relevant structures must be readily abstracted from hundreds and potentially thousands of notes. This imposes a substantial cognitive burden on users and significantly discourages complexity and revision. The present disclosed user interface improves over these conventional approaches.

[0153] Poses - Approach: The “poses” of the disclosed user interface

[0154] The user interface may be configured to allow users to create “poses”, or clips of logic which represent a musical transformation that can target a tree (e.g., the family of a track). When scheduled within a track, a pose may transpose the track’s scale, patterns, and descendants. Poses can include one or more of: a “vector”, a set of transpositions along any scales of the track’s hierarchy; a “voice leading”, a transformation that can move any set of pitches to any other set of pitches; or a set of miscellaneous operations like retrograde, inversion, and phase. A pose can have a temporary or an indefinite duration, and multiple poses can be stacked to sum their effects together.

[0155] Poses - Implementation of Pose Clips

[0156] Users can create a pose by arranging a new “Pose Clip” in the timeline (e.g., FIG. 12) and double clicking on the clip to edit its properties: a dropdown interface opens with a section for vectors (e.g., FIG.13), a section for voice leadings (e.g., FIG.14), and a section for miscellaneous operations (e.g., FIG.15). Pose Clips may be displayed with a distinct pink color and they can be freely dragged to any track and timepoint, visually updating notes in the timeline as they move.

[0157] FIG.12 shows an example Pose Clip that targets Pattern Track AAA and its Pattern Clip representing New Pattern.

[0158] FIG.13 shows an example transposition section of the dropdown editor of a Pose Clip.244908-0584-6290.1

[0159] FIG.14 shows an example voice leading section of the dropdown editor of a Pose Clip, visible as a 4x3 grid.

[0160] FIG.15 shows an example miscellaneous transformation section of the dropdown editor of a Pose Clip.

[0161] Poses - Implementation of efficient keyboard shortcuts for creating poses

[0162] If a track is selected, there may be keyboard shortcuts available to easily update and create poses. A sequence of shortcuts can be notated or saved by the user and saved or loaded as instructions for composition or performance.

[0163] If the user holds any of the QWERTY keys, they may target a specific scale: Q, W, and E may target the three highest scales of the track, respectively; R may target the intrinsic scale of the track; T may target the chromatic scale; Y may target the octave. In much the same way, users can also bind MIDI controllers to poses.

[0164] If the user presses any of the numbers 1-9, they may create a new Pose Clip at the current time point containing a vector with a corresponding offset of 1 to 9 along all targeted scales. If any Pose Clips are selected, the vectors of the Pose Clips may be updated instead. If the user holds minus or tilde, the offset may be negative. If the user presses 0, the offsets along the targeted scales may be reset in any selected Pose Clips.

[0165] Real-time Plugin

[0166] The user interface may be implementable as a plugin (e.g., VST plugin) that can be used inside DAWs and / or notation programs. Once the plugin is added to a MIDI track or clef, it can interact with the program in two ways: (1) The disclosed user interface can send over notes scheduled in the plug-in and output them in the other program; or (2) the user can design a tree in the plugin window to transform inputted notes — the plugin may assign a function to the note based on the user’s scales (e.g. upper diatonic neighbor), then apply any transpositions specified by the user and generate a new output note.

[0167] The scalar hierarchy can be constructed using either mode of the disclosed user interface timeline, Grid or Tree, with rectangular or circular nodes to represent each track and its scale (e.g., FIG.16). The plugin window may comprise a toggle for the graphical mode, an input box for TreeJS to create new trees (e.g., as described further herein), and / or a canvas that contains the user’s tracks and their associated transformations. The user can set a base transposition for each track, schedule a transposition by typing in their request with a time point specified, and employ efficient keyboard shortcuts for creating and editing poses (e.g., described further herein).254908-0584-6290.1

[0168] It should be seen that the musical result—the process of synthesizing motifs based on scales in a collectional hierarchy, and applying complex transformations to the notes—is the same whether the user creates a tree in the disclosed user interface (e.g., implemented as a full web or computer application) and inputs a MIDI note through a Pattern Track, or whether they create a tree in the plugin and input a MIDI note through a MIDI track.

[0169] FIG.16 shows an example tree created with the VST plugin in Tree View, showing each track’s name, label, and current transposition. Track AAA is mounted onto a MIDI track in the user’s DAW, changing its notes.

[0170] Non-real-time Plugin

[0171] The user interface may be configured for usage in a non-real-time mode, in both a DAW and notation program. Here, the user may highlight a collection of notes (e.g., either in a MIDI track or in music notation) and then open the disclosed user interface in a new window to apply various poses to them, based on a definition of the motif and a hierarchy of musical collections. The transformed notes may then be written back to their original location, as new notation objects.

[0172] The user interface window may be modeled on the motif editors (described in FIGs.9-10), including a graphical representation of the notes with: a tab to edit the note durations and values, a tab to toggle whether notes are played based on an internal clock, a tab to specify dependencies among the notes (i.e. whether one is a diatonic or chromatic neighbor), and / or a tab to transform the notes with user-defined or preset poses, including operations like transposition and voice leading.

[0173] FIG.17 shows an example non-real-time plugin transposes the highlighted stream of notes directly within the notation program after the user makes a selection and confirms their transformations.

[0174] Second Experimental Example Implementation of User Interface

[0175] Provided below is a further discussion of the disclosed techniques including an example experimental implementation of the disclosed techniques as a digital audio workstation. It should be understood that the following examples are provided for purposes of illustration and the disclosure is not limited to these features. Any single feature or combination of the features of this example can be combined with any of the other examples or features provided elsewhere herein.264908-0584-6290.1

[0176] In this section, we present a second example of a user interface (e.g., Harmonia)—the world’sfirst pattern-based digital audio workstation (DAW) with integrated structural harmony. By capturing the architectural frameworks of chords and scales that underpin the Western tradition of music, we can revolutionize how musicians work with patterns and unlock the secrets of transposition. Combining the best features of scorewriters like Sibelius and Finale with the cutting-edge capabilities of modern DAWs such as Ableton Live and Logic Pro, we have found that the disclosed user interface can provide a refreshing and immersive musical experience for users of all levels.

[0177] Introduction

[0178] It is very reasonable to be frustrated as a composer right now. In a rapidly changing world overflowing with technology and innovation, one might naively imagine that the creative process of composing music has become a seamless and intuitive experience. However, a closer look at the trials and tribulations of the contemporary composer will reveal a markedly different reality—one characterized by confusion, frustration, and disillusionment.

[0179] Introduction - A Bleak Digital Landscape

[0180] Firstly, many composers are overwhelmed before they even start. Our current tools for music composition often require users to read an extensive set of manuals and tutorials in order to keep up with a vast amount of information that continues to grow every day. After investing a significant amount of money in instruments and software, the beginner will still need to devote countless hours learning the intricacies of their tools and the nuances of tonal syntax. As they encounter contentious, abstract terms like ”music theory” and ”harmony”, these aspiring composers may easily become intimidated or jaded and quickly lose interest in this new hobby.

[0181] Secondly, many composers frequently struggle at the conceptual level. While these musicians might have an intuitive understanding of sequences and chord progressions, they will have a hard time translating their ideas into a set of discrete notes, even with a solid grasp of harmonic principles. As their music expands and their abstractions become more complex, it will become increasingly challenging to notate every note and all but impossible to visualize the broader architecture of their patterns.

[0182] Lastly, nearly every composer will have tofight off their perfectionistic tendencies that ultimately impede the creative process. It can be far more efficient to focus on the bigger picture initially and refine details later; unfortunately, our existing tools often274908-0584-6290.1emphasize a fanatically precise technique that can easily obscure the expression of emotion. As composers waste precious time navigating through menus and operating various plugins, they may inevitably lose some clarity in their ideas and many will despairingly watch the magic of their music disappear right before their very eyes.

[0183] Introduction - Reimagining the Composer’s Toolkit

[0184] Evidently, there is a need for a new kind of software that can be more closely aligned with the big picture ideas of the contemporary composer. There is a pernicious tendency to infantilize the user in response to their confusion rather than acknowledge the shortcomings of our current frameworks, which are becoming increasingly clear as they remain unaddressed. Currently, there are only a few applications that offer a streamlined, pattern-based environment, and none at all that do so with a practical representation of chordal or scalar harmony. The insider secrets of ”perfect” voice leadings and ”essential” chord progressions have become excessively marketized by corporations seeking to turn a profit, and the burden of learning has beenfirmly placed on the user. Rarely is the composer provided with a scaffolding for their music that can reconcile the intuitive knowledge they might possess from their instrument.

[0185] Fortunately, we have found a promising solution for the bleak digital landscape faced by composers today. By integrating the pattern-based principles of a digital audio workstation (DAW) with an expressive harmonic framework grounded in the tradition of staff notation, we can reinvigorate the user with a familiar andflexible interface that bridges the gap between digital and analog music. By distilling a few essential ideas about composition into a streamlined toolkit of operations, we can successfully reunite the form and style of our music within the workstation. In the following paragraphs, we will delve into the design and implementation of the disclosed user interface: the world’sfirst pattern-based DAW built with structural harmony.

[0186] Principles

[0187] In the words of the electronic pioneer Laurie Spiegel, our experiences with music are fundamentally grounded in configurations of notes called patterns, whether we know them as ”chords, motifs, melodies, rhythms, meters, harmonic progressions, etc. right up to sonatas and symphonies” [8]. More recently, the composer and music theorist Dmitri Tymoczko has argued that musical patterns are hierarchical structures of intervals, proposing that Western tonality can be deconstructed into a ”quadruple hierarchy” of voices moving in chords moving in scales moving in chromatic space

[0011] . It is often the case that our aesthetic284908-0584-6290.1ideals of beauty and sublimity can be aligned with architectural principles of form and structure, and the infamous sonatas of Beethoven and Scarlatti are a quintessential integration of these ideas

[0011] .

[0188] Though many centuries of theorists have thoroughly deconstructed the structure of Western harmony and voice-leading, the difficulty of visualizing and notating it has remained a stubborn challenge for almost every musician

[0010] . Fortunately, using Tymoczko’s quadruple hierarchy, it is now possible to understand a wide range of musical phenomena through the familiar lens of transposition—moving the scale along the chord creates modulation; moving the chord within the scale creates voice leadings; moving the voice within the chord creates motivic development, chordal inversion, and what Tymoczko calls ”structured arpeggiation”

[0011] . It is also especially useful when these motions counteract or nearly counteract each other to create efficient voice leadings, allowing melodic lines to move short distances between distant chords like C major and F# major.

[0189] While many transformations of patterns are intuitive to understand—including but not limited to Spiegel’s examples of repetition, reversal, and combination [8]—the art of transposition can be highly elusive and embedded in ”implicit compositional, improvisational, and perceptual habits that may not be immediately available to conscious reflection”

[0011] . Many musicians can oftenfind ”maximally efficient voice leadings within relative ease”, but rarely will this occur from explicitly formalizing the calculations that are involved within the multi-dimensional musical space [9].

[0190] Compounding the difficulty of parsing several layers of music at once, there will also be inherent limitations within the design of any system for musical notation, which can illuminate certain kinds of transposition and obscure others. The way we visualize our music bears weight on the way we can express it, and most of our software is not attuned to these hierarchical structures in a meaningful way. Instead, the current divide between the staff notation of the scorewriter and the piano roll notation of the digital audio workstation has only increased—creating an unnecessary dichotomy in our perception of composition.

[0191] In order to understand the inspiration behind the disclosed user interface’s characterization as a pattern-based DAW with structural harmony, we will explore these two prevailing techniques of visualizing music—staff notation and piano roll notation—and uncover the difficulty in effectively transposing and otherwise manipulating patterns with our current sets of tools like MuseScore and Ableton.

[0192] Principles - The Importance of Transpositions294908-0584-6290.1

[0193] Before we proceed, it is essential to disambiguate the word ”transposition”. We define three kinds of transposition that are foundational to Western music, resulting from the three types of systematic movement within the quadruple hierarchy (FIG.18).

[0194] FIG.18 shows an example Tymoczko’s model of the quadruple hierarchy

[0011] .

[0195] First, there is chromatic transposition, which shifts a collection of notes by a fixed number of semitones in relation to the chromatic scale. This kind of transposition preserves the harmonic structure and melodic contour of a musical pattern while sliding all of its notes by a constant amount. Chromatic transposition is fundamental in many different genres, and it is embedded in the tradition of chromatic instruments like the piano as well as fretted instruments like the guitar.

[0196] Second, there is scalar transposition, which shifts a collection of notes by afixed number of steps in relation to some scale. This kind of transposition will not preserve the tonality and contour of a musical pattern exactly, but its pitches will stay confined to the notes of the scale. Though the chromatic scale is an example of a scale, it is important to distinguish that the unit of musical distance for scalar transposition.

[0197] It is not always equal to the fundamental unit of a semitone as it is for chromatic transposition. Instead, the operation is typically dependent on using a subset of the chromatic scale as a ”musical ruler” to determine the offset for each note, which can be any number of semitones or microtones

[0010] . One clear example of scalar transposition can be found within the design of keyboard instruments, which distinctively arrange their white keys to facilitate transposition along the C major scale.

[0198] Finally, there is chordal transposition, which shifts a collection of notes by a fixed number of steps in relation to its intrinsic scale. Also known as ”rotation” or ”chord inversion” [8], chordal transposition is a special case of scalar transposition where the relative scale exclusively comprises (e.g., or consists) of the notes of the musical object. Though this operation is typically described in the context of rearranging a chord, it can also be useful for deriving the modes of a scale and mapping out voice leadings. Again, we keep this definition separate from scalar transposition to reflect a traditional musical perspective that employs these techniques differently.

[0199] One advantage in broadly defining transposition is that we can expand the notion of transpositional equivalence to group a lot of structurally related patterns together. By equipping ourselves with multiple musical rulers and not just one, we can be strikingly economical with a single idea and develop a motif in many different ways. Navigating these304908-0584-6290.1three kinds of transposition is of paramount importance for any composer working with harmony and melody, and we shall analyze how these ideas are not evident with current implementations of staff and piano roll notation.

[0200] Principles - Transposing Patterns in Staff Notation

[0201] Staff notation, also known as Western musical notation, has been the predominant system of representing Western music for centuries. With a comprehensive set of symbols and markings, a musical score can provide a clear, visual representation of a composition and convey various elements of a piece like pitch, rhythm, dynamics, and articulation. By arranging notes on a staff comprising (e.g., or consisting) of horizontal lines and spaces, composers can construct a timeline of notes with pitches defined by vertical position and durations defined by visual form (FIG.19).

[0202] FIG.19 shows an example C major chord written in staff notation being transposed a) along the chromatic scale; b) along the C major scale; c) along the C major chord.

[0203] The strengths of staff notation easily lie in its visual elegance and expressiveness. The contours of notes and patterns can become immediately recognizable and the use of evocative text can be synergistic with thematic and gestural ideas. There is a rich, cultural interplay between the music and the score that has endured centuries of pushback with a distinctive form.

[0204] Nevertheless, staff notation exhibits certain limitations. The layout is inextricably bound to a sheet of paper, whether physical or digital, and the simplicity of music can become obscured by cluttered information and confusing page turns. In addition, the use of key signatures is constrained to the Western tradition of music and one cannot efficiently communicate any harmony that does not revolve around a major or minor scale. Consequently, the staff positions only correspond with seven-note diatonic scales, so transposing a pattern up one step along a scale of some other size (for instance 5, 6, or 8 notes) does not necessarily align with the next line or space on the score. Simply knowing the key signature of a piece is not enough to know how to manipulate a pattern within the harmony it describes.

[0205] Contemporary scorewriters like Sibelius, Finale, and MuseScore have significantly impacted the process of creating and editing scores by offering a range of powerful tools that promote creativity and facilitate collaboration. Despite these advancements, the programs are still restricted by the traditional conventions of staff notation,314908-0584-6290.1reluctant to push our engagement with the score. Adhering to a rigidly graphical purpose, these popular scorewriters all fail to allow composers to generalize musical objects as patterns of notes.

[0206] Consequently, the user cannot easily edit any of their motivic ideas after they have already been notated. Should a composer choose to edit a single note within one of their recurring patterns, they must go through their entire score and update every instance of it in their music. Not only is this difficult to manage as a piece grows longer, but it can also be exceptionally hard to recognize a pattern after it has been transposed and manipulated in deliberately complex ways. Because of this, the scorewriter is becoming increasingly obsolete in the wake of highly efficient tools like the digital audio workstation.

[0207] Principles - Transposing Patterns in Piano Roll Notation

[0208] Piano roll notation presents a visually simpler alternative to staff notation, often used in conjunction with the MIDI (Musical Instrument Digital Interface) protocol that defines notes by pitch, length, and velocity [7]. Originating from player piano rolls, this graphic representation of music is particularly well-suited for use within digital audio workstations (DAWs). Composers typically create a sequence of notes on a grid-based timeline by following a horizontal axis representing time and a vertical axis representing pitch (FIG.20). In addition, a virtual piano keyboard will often be superimposed on this grid to facilitate note placement and identification.

[0209] FIG.20 shows an example C major chord written in piano roll notation being transposed a) along the chromatic scale; b) along the C major scale; c) along the C major chord.

[0210] Based on a tradition in digital audio rather than the Western score, the piano roll offers users an intuitive, neutral interface that effectively quantizes pitch and duration. Its flattened design allows for a more compact representation of music, while the visually consistent spacing of notes can make chromatic transposition easily recognizable. Moreover, the format’s inherent compatibility with MIDI can make musical ideas highly accessible and distributable. Where staff notation is organized around the diatonic scale, the piano roll is organized around the chromatic scale.

[0211] However, piano roll notation lacks a system for expressing and conveying the harmonic relationships between notes, which can make it challenging to work with voicings, chords, and scales. The difficulty in quickly inferring harmonic significance from the timeline can obscure a composer’s intended tonality and make it all but impossible to recognize324908-0584-6290.1certain kinds of transpositions. Although the design of the piano roll is simple and accessible, it has yet to achieve the same degree of clarity and ubiquity that the score has achieved after centuries of cultural and pedagogical hegemony.

[0212] While most DAWs like Ableton and Reaper provide versatile piano roll interfaces, only some like FL Studio allow users to create and edit patterns. In an effort to accommodate traditional musicians, some DAWs like Logic Pro may offer features such as automatic chord detection or score views of MIDI data. Nevertheless, these tools do not replace the need for a structural implementation of harmony that can formalize the movements of voicings and scales. By relying on users to visualize the hierarchical structures of their music, the digital audio workstation becomes increasingly foreign to both traditional and contemporary tonal composers.

[0213] Approach

[0214] The design of the disclosed user interface (e.g., Harmonia) comes from a pattern- based philosophy for composition that is deeply rooted in the structured harmony of Western music. By combining the most effective features of scorewriters and DAWs, we can create a novelly hybridized software that revolutionizes the way composers can interact with chords and scales.

[0215] Approach - Building A Pattern-Based DAW

[0216] The disclosed user interface may be modeled as a digital audio workstation, dividing the interface into a score-based editor for notation and a multi-track sequencer for arrangement. This design allows the user to easily distinguish between the conceptual and constructional goals of composition while still maintaining the ability to integrate them together. The score-based editor may offer a robust set of controls for crafting patterns or scales with staff notation, while the multi-track sequencer may provide a grid-based timeline for arranging and developing musical motifs with a streamlined toolkit of operations. After creating a pattern or choosing from a variety of presets, the user can select from an extensive library of instruments and effects to shape the timbre of each part. With orchestration and playback built into the system, the disclosed user interface may capture the live and interactive experience of composition.

[0217] Approach - Approaching Scalar Harmony

[0218] In order to connect a scalar collection of notes with one or more patterns of notes, the disclosed user interface may introduce a Scale Track and / or the Pattern Track (FIG.21).334908-0584-6290.1Building upon existing classifications like the MIDI Track and Audio Track, the Scale Track may define the scale for the patterns played by its associated Pattern Tracks.

[0219] FIG.21 shows an example Pattern Track nested within a Scale Track.

[0220] A pattern track may define a motive as existing within a particular hierarchical nesting of scales. Notes can either be defined directly as scale degrees (e.g. “scale degree 60 of level 0 in the hierarchy”) or relatively (e.g. “start at scale degree 60 of level 0 in the hierarchy, calculate the scale-degree value at level 1 in the hierarchy, and then go up one step in the level-1 collection.”). This design grants the Pattern Track a distinct harmonic identity, allowing for a logical management of tonality across different Scale Tracks in the project.

[0221] Approach - Embedding the Quadruple Hierarchy

[0222] Rather than moving linearly through MIDI pitch space, the disclosed user interface allows for navigation of pitch space by adopting Tymoczko’s quadruple hierarchy, modelling voices moving along chords moving along scales moving along chromatic space. Transposition may be defined as a three-dimensional operation Ni· Tj· tk, corresponding to chromatic (N), scalar (T), and chordal (t) components consistent with the quadruple hierarchy.

[0223] FIG.22 shows an example C major chord written in the disclosed user interface being transposed along i) the chromatic scale; ii) the C major scale; and iii) its intrinsic scale.

[0224] A transposition may be applied to a specific beat in either a Scale Track (e.g., transposing its scale) or a Pattern Track (e.g., transposing its patterns). The transposition may persist until a new transposition is specified. To avoid cluttering the interface, a transposition may be notated on the timeline using a pink, wand-shaped icon (FIG.22).

[0225] With thisflexible and innovative approach to transposition, the disclosed user interface significantly reduces the cognitive load of managing individual notes and remembering past operations. While the concept of multidimensional coordinates may initially seem unintuitive, the user is now free to explore transposition, modulation, voice leading, and all of the phenomena that are possible with a structural approach to music.

[0226] Composing

[0227] In this chapter is described Scriabinism—a musical example that was created with the disclosed user interface—and analyze why the program was so helpful in composing it. We will conduct a formal evaluation of the website later, but it is important to understand the principles of composing with disclosed user interface before we approach its implementation.344908-0584-6290.1

[0228] Scriabinism is a groovy piece of house music featuring a melody directly inspired by the Presto con allegrezza theme from Scriabin’s Sonata No.5 (e.g., FIG.23). Though the tune can be easily hummed, the theme is rather musically intricate, comprising (e.g., or consisting) of a syncopated chordal riff harmonized by a dominant eleventh chord in F# major. The composer originally abandoned the piece due to the difficulty of maneuvering the intricate voicing around the six accidentals of the F# major scale—and it was only revived three months later because the disclosed a version of the user interface use for experimentation allowed the composer to much more efficiently manipulate the composition.

[0229] FIG.23 shows an example main four-bar motif of Scriabinism in F# major.

[0230] With the ability to instantly derive many compelling inversions and transpositions, the disclosed user interface made it possible to focus on developing the color of the theme rather than working out the chordal and scalar logic. The riff was notated as four patterns in the disclosed user interface—one for each clef in each cell—and arranged within an F# Major Scale Track; the right hand is played with a Wurlitzer Pattern Track and the left hand is played with an Electric Organ Pattern Track.

[0231] After the melody is introduced for eight bars, it is accompanied by an energetic, four-on-the-floor beat; within a separate Chromatic Scale Track, we create a drum sequence from a Kick Pattern Track, a Snare Pattern Track, and a Hi-Hat Pattern Track. After repeating the motif twice more, afirst transformation is introduced that modulates the F# major scale down byfifth to B major (e.g., FIG.24).

[0232] FIG.24 shows an example transposition of the music shown in FIG.23. In the first cell, the right hand is transposed by (N0,T3,t0) while the left hand is transposed by (N0,T0,t3). In the second cell, the right hand is continued while the left hand is transposed back to (N0,T0,t0).

[0233] FIG.25 show an example user interface (e.g., Harmonia) showing the transposed motif from FIG.24, applying the transpositions to the original pattern and scale.

[0234] Initially, we identified thefirst two chords on the piano to start the pattern with a C# minor voicing, but it was not clear how tofill in the rest of the motif while it was being both transposed and modulated. After bringing the Scale Track down to B major in the disclosed user interface, it did not take long tofind that T3 and t3 would get the right and left hand where they belonged, respectively; and thus, the rest of the transposed pattern was automaticallyfilled in without a second thought (e.g., FIG.25).354908-0584-6290.1

[0235] This variation is repeated twice, and then a second development modulates the melody back to F# major while transposing the right hand up one step along the chord (e.g., FIG.26). Though it might sound easy to invert a voicing with a scale of four notes, it does not necessarily produce an intuitive transformation. Just by inverting the right hand up one step, the harmony of thefirst cell changes from outlining onlyfifth and sixth intervals to outlining fourths,fifths, sixths, and sevenths, with an inner minor second voicing that was not present before.

[0236] FIG.26 shows an example musical composition in which the right hand of the motif is transposed by (N0,T0,t1).

[0237] FIG.27 shows an example user interface (e.g., Harmonia) show thefirst half of the transposed motif from FIG.26.

[0238] When working this out at the piano, it was tempting to preserve the same hand shapes of thefifth and sixth chords, but this would have lacked any grounding in the structure of the underlying scale. A similar situation arises when transposing the second cell of the motif: the last G# note of the original motif seemed distinct from the Bmaj7 chords that preceded it, but we see after inverting the cell that the chords now have a G# in them inherited from the melody. Keeping track of different intrinsic scales takes a lot of effort and adding a single pitch to a cell can significantly changes its transpositions. Fortunately, with the disclosed user interface, we were able to completely skip the work of doing these calculations and effortlessly obtain this colorful inversion of the original idea (e.g., FIG.27).

[0239] Upon adding an Electric Bass Pattern Track with a randomized 8-note bassline into the F# Major Scale Track, the energy of the piece became unstoppable. As the development of the original motif continues, the bass is paired with its own set of transpositions that soar along the chord and scale. After a hair-rising crescendo leads us to the end of the piece, the composition smoothly loops back to its beginning and eagerly starts anew.

[0240] Accounting for all of the transpositions in this piece, it would probably have taken a considerable amount of time to write down every note—especially as the music rapidly moves through register and harmony. It is quite a challenge to apply these different operations independently on different voicings, and all but impossible to remember them later on in the distant future. Now, with the disclosed user interface’s framework of structural harmony, we can safely explore our ideas with an infinite number of variations readily available.364908-0584-6290.1

[0241] Implementation

[0242] In this chapter, we will provide an extensive overview of an example implementation of the user interface while describing the essential technologies used to power the implementation.

[0243] Implementation - The System Architecture

[0244] One of the benefits of developing a website rather than a standalone piece of software is the ability to utilize a wide range of pre-existing web libraries ranging in purpose from state management to browser audio to staff notation. In order to implement a digital audio workstation, it was important to design a robust system architecture supported by cutting-edge, state-of-the-art libraries.

[0245] Implementation - A Reactive Core with Typescript

[0246] React is one of the most popular frameworks used today for developing web interfaces and it is backed by an incredibly active community of millions of developers [2]. The exceptional performance andflexibility of the library made it the obvious choice for building scalable components that could adapt as features were implemented and removed. Instead of using a traditional JavaScript environment, we pair React with Typescript to enforce strict typing and work with essential, user-defined types like Scales and Patterns [5].

[0247] Implementation - State Management with Redux

[0248] Redux is a state management library designed for centralizing all information within a global ”store” [3]. The Redux team emphasizes many principles that are important for software development, such as maintaining aflattened and normalized state, and they offer numerous performance optimizations like memoization. The Redux store is the foundation of the disclosed user interface and it manages all of its serializable data, including the global collection of scales, patterns, tracks, and transpositions. By making use of redux- undo, we successfully implemented support for undo and redo operations.

[0249] Implementation - Browser Audio with Tone.js

[0250] Tone.js is a Web Audio framework for creating interactive music in the browser and it is responsible for synchronizing and scheduling all audio in the disclosed user interface [4]. With its common DAW features and high-performance building blocks, Tone.js was invaluable in offering a wide variety of musical objects and types. To play a pattern from a track, each Pattern Track uses an instrumental sampler (Tone.Sampler) loaded with orchestral samples from Ableton Live. The Grand Piano serves as the default instrument for every374908-0584-6290.1Pattern Track, with a total of, for example, 50 instruments spanning seven distinct categories (e.g., Additional Information - Section C).

[0251] Each instrument may be connected to a channel strip interface (e.g., Tone.Channel) that manages its volume, pan, mute, and solo. All of these channels may then be routed into a main audio bus that lets individual tracks be muted, soloed, and controlled by a global volume. To provide the user with more timbral customization, we chain each instrument with a series of audio effects, including warp (Tone.PitchShift), reverb (Tone.Reverb), chorus (Tone.Chorus), and delay (Tone.FeedbackDelay). Every effect comes with a set of three adjustable parameters that can be controlled by a rotary knob, one of which is always the wet parameter that controls how much of the effected signal should come through.

[0252] Implementation - Styling with TailwindCSS

[0253] TailwindCSS is a utility-first CSS framework that provides many pre-built styling classes for building responsive and customizable interfaces. By leveraging its color palette and visual consistency, we were able to create an appealing user interface inspired by DAWs like Ableton and Reaper with modernflashes of color.

[0254] Implementation - The User Interface

[0255] The user interface may comprise (e.g., or consists of ) three central components— the Timeline, the Editor, and the Toolbar—which facilitate various interactions within the website.

[0256] FIG.28 shows an example Timeline of the disclosed user interface, modelled on the DAW.

[0257] The example Timeline (FIG.28) may be a multi-track sequencer for scheduling audio events (e.g., where patterns can be arranged as clips to be played within a specific scale). Adopting the familiar design of a DAW, the user interface may comprise a vertical panel of Scale Tracks and Pattern Tracks displayed alongside a horizontal grid of cells.

[0258] Using react-data-grid

[0259] After experimenting with different libraries, react-data-grid was ultimately chosen to implement the Timeline due to its lightweight utility and performance [6]. The grid can be ”virtualized”, meaning that it will only render cells that fall within the current view of the user’s screen. By utilizing a minimal set of dependencies, this optimization allows the Timeline to efficiently update thousands of cells every beat without expensively rendering all of them.384908-0584-6290.1

[0260] Scale Tracks

[0261] A Scale Track can be added by clicking the corresponding button at the bottom of the track list. Each Scale Track may include a button to change its scale (e.g., which opens the Scale Editor), a button to add a nested pattern track, and a dropdown menu for copying, clearing, and deleting the track. If not explicitly named, a Scale Track’s name will default to its scale, dynamically updating as the scale undergoes transposition.

[0262] Pattern Tracks

[0263] A Pattern Track can be added by clicking the corresponding button in a Scale Track. Each Pattern Track features a button to change its instrument (which opens the Instrument Editor), as well as volume, mute, and solo controls, and a dropdown menu for copying, clearing, and deleting the track. If not explicitly named, a Pattern Track’s name will default to its instrument.

[0264] Pattern Clips

[0265] A Pattern Clip can be added by toggling the Brush Icon in the Toolbar and selecting a beat within a Pattern Track in the Timeline. Clips can be selected either individually or as a group, and then moved, copied, or deleted within the Timeline (e.g., Addition Information Section E). Additionally, the user can apply various operations to transform the clips, which will be further explained in the Toolbar section.

[0266] Transpositions

[0267] A Transposition can be added by toggling the Wand Icon in the Toolbar and selecting a beat within either a Scale Track or a Pattern Track on the Timeline, after specifying the desired transposition. Once added, the user can delete the Transposition by clicking on it or view its coordinates by hovering over it.

[0268] Implementation - Interacting with the Editor

[0269] FIG.29 shows an example Pattern Editor.

[0270] The Editor may comprise a dedicated window used for designing patterns (e.g., FIG.29), scales (e.g., FIG.30), and instruments (e.g., FIG.31). The Editor may provide a streamlined interface with clear, robust controls and thoughtfully curated presets. Once invoked, the Editor will display over the Timeline and occupy the main view until it is dismissed.

[0271] Using OpenSheetMusicDisplay

[0272] OpenSheetMusicDisplay (OSMD) is a web library designed for rendering sheet music in the MusicXMLfile format for Western musical notation [1]. By developing a394908-0584-6290.1custom serializer to convert musical objects into MusicXML, we were able to use OSMD to display scales and patterns as sheet music, while also providing the ability to export them individually as MusicXMLfiles. Additionally, we succeeded in integrating a cursor for the score in the Pattern Editor, allowing the user to select individual notes with precise control, though this feature is buggy with larger patterns.

[0273] Pattern Editor

[0274] The Pattern Editor (e.g., FIG.29) can be opened by clicking the Pencil Icon in the Toolbar next to the currently selected Pattern. The interface may provide an interactive editor for creating custom patterns, as well as a categorized list of preset patterns to choose from (e.g., Additional Information - Section A). Each Pattern Clip on the Timeline corresponds to a pattern found in the Pattern Editor. The interface comes with a virtual piano paired with three types of controls: general actions, note actions, and transformations.

[0275] General Actions: The user can undo / redo the most recent changes made to a pattern, play a preview of the pattern, start adding it as a Pattern Clip in the Timeline, or export the pattern to a MusicXMLfile. The user can also toggle a cursor for the score to navigate the pattern and edit or remove specific notes.

[0276] Note Actions: The user can add notes to their custom pattern by toggling the corresponding button, selecting their desired note duration, and playing the virtual piano. The user can hold Shift while adding notes to create a chord, or press a button to add a rest based on the currently selected note duration. If the cursor is highlighting a specific chord, the user can update the notes of the chord, insert a new note or rest directly after it, or erase it from the stream entirely. The user can also immediately clear the entire pattern.

[0277] Transformations: The user can transpose the pattern along the chromatic scale or invert it along its implicit scale. The user can also contract and expand the pattern (halving and doubling all note durations), as well as slice and repeat the pattern (halving or doubling the number of notes). Lastly, the user can shuffle the notes of the pattern or generate a random eight-note melody.

[0278] Scale Editor

[0279] The Scale Editor (e.g., FIG.30) can be opened by clicking the corresponding button in a Scale Track. The interface provides an interactive editor for customizing the scale of the currently selected Scale Track, as well as a categorized list of preset scales to choose from (e.g., Additional Information - Section B). Additionally, the user can save the current scale as a custom scale that will be available for every Scale Track to use.404908-0584-6290.1

[0280] FIG.30 shows an example Scale Editor.

[0281] The interface may come with a virtual piano paired with three types of controls: general actions, note actions, and transformations.

[0282] General Actions: The user can undo / redo the most recent changes made to a scale, play a preview of the scale, or export the scale as a MusicXMLfile.

[0283] Note Actions: The user can add or remove notes by toggling the respective button and playing the virtual piano, or instantly clear all notes from the scale.

[0284] Transformations: The user can transpose the scale along the chromatic scale or transpose it along its intrinsic scale.

[0285] Instrument Editor

[0286] The Instrument Editor (e.g., FIG.31) can be opened by clicking the corresponding button in a Pattern Track. The interface contains an interactive panel of rotary knobs for customizing the instrument of the currently selected Pattern Track, as well as a categorized list of instruments to choose from (e.g., Additional Information - Section C). The editor comes with a virtual piano that can be played to preview the timbre of the instrument.

[0287] FIG.31 shows an example Instrument Editor.

[0288] Warp: The user can set the Wet (e.g., signal wetness), Pitch (e.g., offset in semitones), and Window (sample length in seconds) parameters of the Warp Effect.

[0289] Reverb: The user can set the Wet (e.g., signal wetness), Decay (e.g., duration in seconds), and Predelay (e.g., duration in seconds) parameters of the Reverb Effect.

[0290] Chorus: The user can set the Wet (e.g., signal wetness), Depth (e.g., chorus depth), and Delay (e.g., duration in milliseconds) parameters of the Chorus Effect.

[0291] Delay : The user can set the Wet (e.g., signal wetness), Time (e.g., delay time in seconds), and Feedback (e.g., delay feedback) parameters of the Delay Effect.

[0292] Implementation - Making Use of the Toolbar

[0293] FIG.32 shows an example Toolbar split into two rows.

[0294] The Toolbar (e.g., FIG.32) may comprise a streamlined toolkit of controls designed for managing the project, playing audio, and manipulating patterns. It may always be visible above the Timeline and organized into four distinct sections: File Control, Audio Transport, Pattern Toolkit, and Settings / Resources.

[0295] File Control (e.g., FIG.33): The user can click the File Icon to toggle a menu for naming and saving thefile, loading a custom or presetfile, and clearing the project after414908-0584-6290.1confirmation. There is also an undo and redo button for actions made to the Timeline (i.e. tracks, clips, and transpositions).

[0296] FIG.33 shows an example File Control section of the Toolbar.

[0297] Audio Transport (e.g., FIG.34): The user can view the current time displayed in the bars:beat:sixteenths format and click one of three buttons to stop, play / pause, or loop / unloop the global transport. While looped, the relevant section of the Timeline may be annotated in purple. Additionally, the user can seek to a specific time by clicking on a beat in the Timeline.

[0298] FIG.34 shows an example Audio Transport section of the Toolbar.

[0299] Pattern Toolkit (e.g., FIG.35): The user can click on the currently selected pattern to select from a categorized dropdown list of all patterns or toggle the Pattern Editor by clicking on the pencil-shaped icon next to it. Additionally, they can engage withfive essential operations for manipulating patterns:

[0300] FIG.35 shows a Pattern Toolkit section of the Toolbar.

[0301] Sequencing (e.g., Brush Icon): The user can add a Pattern Clip to a specific beat falling under a Pattern Track on the Timeline.

[0302] Fragmentation (e.g., Scissors Icon): The user can slice a Pattern Clip at a specific beat to extract two sub-patterns on the Timeline.

[0303] Combination (e.g., Link Icon): The user can merge one or more Pattern Clips to create a new Pattern Clip on the Timeline.

[0304] Repetition (e.g., Clock Icon): The user can repeat one or more Pattern Clips on the Timeline by a desired number of times.

[0305] Transposition (Wand Icon): The user can add a Transposition to any beat in a track within the Timeline after specifying a chromatic, scalar, and chordal offset.

[0306] Evaluation

[0307] In this section, we will assess the performance and effectiveness of this disclosed implementation of the user interface (e.g., Harmonia) with two main approaches: a user- based evaluation describing the results from our focus group and an heuristic evaluation of the website using Nielsen’s 10 usability heuristics. This analysis will highlight the strengths and weaknesses of the software while potentially identifying areas for future improvement.

[0308] User Evaluation

[0309] An interactive focus group was conducted to gather user feedback on the disclosed user interface, pollingfifteen juniors and seniors from Princeton University. The424908-0584-6290.1session included a diverse range of participants with a varying level of musical experience: some were classical, jazz, and electronic musicians who were already familiar with a DAW, and some were poets, journalists, and engineers who had never interacted with any kind of compositional software before. The meeting was held informally in a relaxed setting to foster a stress-free environment that was comfortable and conducive for composition. After an introductory presentation framing the bleak digital landscape faced by current musicians, the participants were given a brief tutorial on how to use the disclosed user interface and instructed to create a sample composition. At the end of the session, users were encouraged to share their music and asked to reflect on their experience using the software. Upon analyzing the focus group feedback, we found several overarching themes and trends that emerged related to functionality, accessibility, and education.

[0310] User Evaluation - Response to Functionality

[0311] There was a positive feedback to the functionality of the website and most users felt that they were provided with a comprehensive toolkit of features. The musicians of the group were excited to work with an explicit representation of harmony and eager to explore the use of transpositions beyond a surface level, but some of the participants with less musical experience were confused about the relevance of scales or the key differences from DAWs like GarageBand. Overall, there were no issues detected with the website except for one potentially fatal bug related to inputting an invalid BPM value (which was quickly resolved after the focus group) and participants were usually able to answer their own technical questions after some quick realization. Upon composing a piece of music, the participants were asked if they had any suggestions or ideas for the project. A new kind of rhythmic track was proposed to differentiate between pitched and unpitched percussive instruments, which would be plausible to implement in the near future either as is or perhaps as a toggled option for Pattern Tracks. Another potential improvement would be additive transpositions that could compound operations; for instance, to create a continually descending bassline for a piece of algorithmic music.

[0312] User Evaluation - Response to Accessibility

[0313] There was a strongly enthusiastic response to the user interface of the software and many participants noted its aesthetically appealing and inviting design. There were no objections to any font or color palette choices, but one user found a few icons that they thought could benefit from a simpler design. Though they were initially selected to be visually engaging, the icons were ultimately more confusing than illustrative and thus434908-0584-6290.1simplified accordingly after the focus group. In addition, we noticed that there were a number of participants who initially struggled to differentiate between the two kinds of tracks, and some who faced difficulty in navigating between the Timeline and the Editor. After some reflection, the track controls were redesigned to include labeled buttons for the most important commands, such as creating a Pattern Track from a Scale Track, and a simple dropdown menu for general track commands.

[0314] One user was immediately excited by the possibility of having the website available as a responsive or dedicated mobile application that they could use instead of having to hum a tune into their phone while on public transit. The website is currently unoptimized for mobile devices, largely due to the difficulty of adapting its intricate functionality to a significantly smaller viewport, but this will certainly be attempted in the future. Some users were open to the idea of using the disclosed user interface as a VST plugin for their DAW, but they unanimously preferred the accessibility of a website if it could be integrated with MIDIfiles. While this feature remains a major goal, the electronic musicians looked forward to future cross-compatibility with their DAWs.

[0315] User Evaluation - Response to Educational Value

[0316] Quite interestingly, there was a surprisingly strong reception towards the educational value of the software. Participants with varying musical experience felt that the presets were very well-curated and many reported that they learned a new scale or instrument while working with the software. Even those with a limited musical background felt empowered by the familiar options to produce a composition that they were proud of. One musician was particularly appreciative that they were able to reconcile their ability to read staff notation with the technicalities of a DAW, allowing them to focus on composing their music instead offiguring out the controls. Another participant cheerfully exclaimed that ”it’s bongo time!"

[0317] A number of musicians saw potential in using the website to teach music theory and many felt that they would have benefited from learning about harmony with such a tool. This would be a very compelling direction for the future of this project and it is exciting to imagine the website as an academic platform for theoretical lectures or compositional assignments. The lack of standardized digital applications within thefield of music education suggests the need for innovation, and it will definitely be interesting to explore the long-term potential of the disclosed user interface as a tool for both creative and educational purposes.

[0318] Heuristic Evaluation444908-0584-6290.1

[0319] We evaluate the user interface of the disclosed user interface with Nielsen’s 10 usability heuristics, a widely accepted set of guidelines for user interface design.

[0320] Heuristic Evaluation - Visibility of system status

[0321] The disclosed example user interface may provide users with clear visual indicators of the system status through the use of descriptive labels and strategic coloring (e.g.filling or animating a button to represent a consistent screen state). There is no information about the Timeline or the Editor that the user cannot infer from afirst glance, and all interactions with the interface give appropriate feedback within a reasonable amount of time.

[0322] Heuristic Evaluation - Match between system and the real world

[0323] The software may use familiar language and basic terminology to ensure that users with any level of music experience are able to understand the website. Originally, there was a considerably more technical vocabulary containing terms like ”System” (for a hierarchical system of scales) and ”Coll” (for a patterned collection of notes), but it was updated to simplify the foundational concepts and adopt simpler expressions like ”Scale” and ”Pattern” in order to ground it in the real world.

[0324] Heuristic Evaluation - User control and freedom

[0325] The disclosed example user interface may support the ability to easily undo and redo actions, providing a significant degree offlexibility and support when composing. The user can control the most recent changes occurring at three different levels of the application: the Scale Editor, the Pattern Editor, and the Timeline. Furthermore, there are often multiple pathways for executing certain actions, such as by using a menu button or a keyboard shortcut.

[0326] Heuristic Evaluation - Consistency and standards

[0327] The user interface maintains consistency in its design with predictable vocabulary, inconspicuous typography, and distinctive iconography. By adhering to established conventions of music notation software and building upon modern software designs with a compelling color palette, there is a familiar yet novel experience for users coming from either the DAW or the scorewriter.

[0328] Heuristic Evaluation - Error prevention

[0329] There are many guardrails for the user as they navigate the software and they are always provided with a confirmation option before clearing their project. Though there are454908-0584-6290.1ways to push the limits of the browser that can lead to distorted or glitched audio, the user is not able to corrupt theirfile or load an incompatiblefile type.

[0330] Heuristic Evaluation - Recognition rather than recall

[0331] By implementing a limited number of overlays and displaying the names of track scales and instruments on the Timeline, we ensure that the user can easily recognize their options and choices rather than having to recall them. Every button is either clearly labeled with text or described by a tooltip triggered by hovering over it.

[0332] Flexibility and efficiency of use. The disclosed user interface may offer various features and keyboard shortcuts that cater to users with different levels of experience (e.g., Additional Information – Section E). It is possible for beginners to compose by exclusively using presets and for professionals to avoid them entirely. There is still room for improvement with customizable key bindings, but the current options provide exceptional flexibility and efficiency for composing with harmony.

[0333] Heuristic Evaluation - Aesthetic and minimalist design

[0334] The design of the disclosed user interface may be aesthetically pleasing and deliberately minimalist. The interface clearly highlights the most important tools for the user while avoiding any unnecessary information that can clutter the screen. After updating some of the website’s iconography in response to feedback from our focus group, we have ensured that every element is easily recognizable and visually consistent.

[0335] Heuristic Evaluation - Help users recognize, diagnose, and recover from errors

[0336] Though there are relatively few errors for the user to encounter in the disclosed user interface, one of the benefits of a web-based application is that it is always possible for the user to refresh their page or clear their cache should a problem occur. However, it is difficult to clearly communicate this information within the software, so it will be important to create a support page as the capabilities of the website grow.

[0337] Conclusion

[0338] As the world’sfirst pattern-based DAW with structural harmony, the example implementation of the disclosed user interface is a promising and potentially groundbreaking solution for the bleak digital landscape faced by the contemporary composer. By formalizing the hierarchical structures of Western music, we have created an innovative piece of software that revolutionizes how composers can work with scales, patterns, and transpositions.

[0339] Further improvements

[0340] We identify three primary areas of improvement:464908-0584-6290.1

[0341] Expressiveness: The composition of patterns can still become more expressive with key features like note velocities and tuplet rhythms.

[0342] Performance: The performance issues in larger projects will eventually need to be addressed through code optimizations.

[0343] Responsiveness: The website may be difficult to adapt to mobile devices, yet its responsiveness remains important for increasing accessibility.

[0344] The Platform or the Plugin

[0345] Though there are still improvements to be made, the next steps of the disclosed user interface look very promising and we see two clear directions for the future of this project:

[0346] The Platform: On the one hand, there is the opportunity to expand the website and create a centralized platform for composition. Given an effective branding, there is a long- term potential to reach millions of users with community-driven features such as an integrated marketplace of scales and patterns. By leveraging the reach of the browser, this approach could be widely popular and innovative.

[0347] The Plugin: On the other hand, there is the opportunity to pivot the website into a plugin that can be used within existing tools like the DAW. Adopting support for MIDI, we can integrate the disclosed user interface within the workflows of current users and support a wide range of instruments and effects. By liberating the website from the browser, this approach could be directly profitable andflexible.

[0348] Regardless of the direction chosen, we hope that the ideas presented here will proliferate through continued refinement and leave a lasting impact on the design of future music and music systems. As we venture forward into an era with artificial intelligence that may potentially overshadow human musicians, it is important that we reassess the current landscape of music composition and empower the next generation of composers to express their art with clarity, confidence, and boundless creativity.

[0349] Additional Information

[0350] The following sections provide additional information relevant to the disclosed user interfaces.

[0351] Additional Information - Section A

[0352] Preset Patterns

[0353] The disclosed user interface may comprise and / or be configured to perform any one or any combination of the following preset patterns available to every user, distributed across474908-0584-6290.1three harmonic, three melodic, and four rhythmic categories: Basic Chords, Extended Chords, Famous Chords; Basic Patterns, Extended Patterns, Famous Patterns; Basic Durations, Simple Rhythms, Latin Rhythms, and Clave Patterns.

[0354] Basic Cords: Major Chord, Minor Chord, Diminished Chord, Augmented Chord, Major Seventh Chord, Minor Seventh Chord, Dominant Seventh Chord, Diminished Chord.

[0355] Extended Chords: Major Ninth Chord, Minor Ninth Chord, Dominant Ninth Chord, Major Eleventh Chord, Minor Eleventh Chord, Dominant Eleventh Chord, Major Thirteenth Chord, Minor Thirteenth Chord, and Dominant Thirteenth Chord.

[0356] Famous Chords: Tristan Chord, Mystic Chord, Elektra Chord, Farben Chord, Purple Haze Chord, So What Chord, Bond Chord.

[0357] Basic Patterns: Major Arpeggio, Minor Arpeggio, Diminished Arpeggio, Augmented Arpeggio, Major Seventh Arpeggio, Minor Seventh Arpeggio, Dominant Seventh Arpeggio, Diminished Seventh Arpeggio.

[0358] Extended Patterns: Major Ninth Arpeggio, Minor Ninth Arpeggio, Dominant Ninth Arpeggio, Major Eleventh Arpeggio, Minor Eleventh Arpeggio, Dominant Eleventh Arpeggio, Major Thirteenth Arpeggio, Minor Thirteenth Arpeggio, Dominant Thirteenth Arpeggio.

[0359] Famous Patterns: Prelude in C, Alberti Bass, Turkish March, Fate Motif, Revolutionary Etude, Zarahustra Fanfare, The Lick, Happy Birthday.

[0360] Basic Durations: Whole Note, Half Notes, Quarter Notes, Eighth Notes, Sixteenth Notes.

[0361] Simple Rhythms: Dotted Whole Note, Dotted Half Notes, Dotted Quarter Notes, Dotted Eighth Notes, Eighth + Two Sixteenths, Two Sixteenths + Eighth, Dotted Eighth + Sixteenth, Sixteenth + Dotted Eighth, Sixteenth + Eighth + Sixteenth.

[0362] Latin Rhythms: Habanera, Tresillo, Cinquillo, Baqueteo, Cascara, Montuno.

[0363] Clave Patterns : 3-2 Son Clave, 2-3 Son Clave, 3-2 Rumba Clave, 2-3 Rumba Clave, 3-2 Bossa Nova Clave, 2-3 Bossa Nova Clave

[0364] Additional Information - Section B

[0365] Preset Scales

[0366] The disclosed user interface may comprise and / or be configured to perform any one or any combination of the following unique preset scales (e.g., There are 44 scales in total, but we decided to include the Major and Minor scales separately from the Ionian and Aeolian scales for ease of accessibility) available to every user, distributed across seven categories related to484908-0584-6290.1scale size and familiarity: Basic Scales, Diatonic Scales, Pentatonic Scales, Hexatonic Scales, Heptatonic Scales, and Octatonic Scales, and Uncommon Scales.

[0367] Basic Scales: Chromatic Scale, Major Scale, Minor Scale, Pentatonic Major Scale, Pentatonic Minor Scale, Blues Scale.

[0368] Diatonic Scales: Lydian Scale, Ionian Scale, Mixolydian Scale, Dorian Scale, Aeolian Scale, Phrygian Scale, Locrian Scale.

[0369] Pentatonic Scales: Yo Scale, In Scale, Hirajoshi Scale, Iwato Scale, Insen Scale.

[0370] Hexatonic Scales: Major Hexatonic Scale, Minor Hexatonic Scale, Augmented Scale, Prometheus Scale, Tritone Scale, Whole Tone Scale.

[0371] Heptatonic Scales: Harmonic Minor Scale, Melodic Minor Scale, Harmonic Major Scale, Neapolitan Minor Scale, Neapolitan Major Scale, Altered Scale, Acoustic Scale.

[0372] Octatonic Scales: Bebop Major Scale, Bebop Dorian Scale, Bebop Harmonic Minor Scale, Bebop Melodic Minor Scale, Bebop Dominant Scale, Octatonic Scale (W-H), Octatonic Scale (H-W).

[0373] Uncommon Scales: Hungarian Minor Scale, Hungarian Major Scale, Ukrainian Dorian Scale, Persian Scale, Byzantine Scale, Enigmatic Scale.

[0374] Additional Information - Section C

[0375] Preset Instruments

[0376] The disclosed user interface may comprise and / or be configured to perform any one or any combination of the following preset instruments available to every user, distributed across seven categories: Keyboards; Guitar and Bass; Strings; Brass; Woodwinds; Mallets; and Percussion. Under each category, we list the name of every related instrument along with the name of its corresponding Ableton Pack.

[0377] Keyboards: Grand Piano, Electric Piano, Rhodes Piano, Wurlitzer, Clavichord, Church Organ, Electric Organ, Layered Piano.

[0378] Guitar and Bass: Electric Guitar, Muted Guitar, Lead Guitar, Tremolo Guitar, Electric Bass, Sub Bass, Wobble Bass, Octave Bass.

[0379] Strings: Violin, Viola, Cello, Double Base, String Ensemble.

[0380] Brass: Trumpet, Trombone, Tube, French Horn, Saxophone, Brass Ensemble

[0381] Woodwinds: Flute, Clarinet, Oboe, Bassoon, Piccolo, Woodwinds Ensemble.

[0382] Mallets: Marimba, Xylophone, Vibraphone, Glockenspiel, Timpani, Tubular Bells.494908-0584-6290.1

[0383] Percussion: Kick, Snare, Clap, Tom, Hi-hat (Open), Hi-hat (Closed), Conga, Bongo, Shaker, Cymbal, Crash.

[0384] Additional Information - Section D

[0385] Musical Demos

[0386] The disclosed user interface may comprise and / or be configured to perform any one or any combination of the following demos available to every user: Harmonium, Spiegelism, Metallurgy, Xylophonia, and Scriabinism. Each piece embodies a different genre, showcasing the versatility of the website with unique approaches to composition.

[0387] Composition 1: Harmonium

[0388] Harmonium is a groovy composition in the key of G major that presents a variety of patterns crafted with different approaches:

[0389] Composing with preset patterns: The four-part drum track was assembled from various rhythmic presets overlapped with each other, comprising (e.g., or consisting) of a hi- hat playing straight sixteenths, a kick playing the tresillo rhythm, a clap playing a baqueteo rhythm, and a shaker playing a 2-3 bossa nova clave.

[0390] Transposing and merging patterns: The main lead was created by composing an eight-note Gmaj13 arpeggio and merging it with two scalar transpositions to produce a I-IV- V harmonic progression.

[0391] Shuffling and randomizing patterns: The bassline was adapted from the lead motif and then shuffled, halved, and repeated to produce many new variations.

[0392] Composition 2: Spiegelism

[0393] Spiegelism is a minimalist composition in the key of C mixolydian that features a deliberate approach to motivic development, inspired by the works of Laurie Spiegel.

[0394] Defining an initial idea: The foundation of the piece was explicitly created from a six-note Bb major arpeggio repeated against an eight-note Csus4maj7 arpeggio for exactly fifty measures.

[0395] Composing with efficient voice leading : Throughout the piece, the two main patterns are transposed along the chord and scale to produce a variety of efficient voice leadings that create compelling melodic lines.

[0396] Exploring color: The music consistently explores various tonalities, with the harmony of the main Scale Track descending byfifths to F mixolydian and Bb mixolydian to contrast with different colors of scales.

[0397] Composition 3: Metallurgy504908-0584-6290.1

[0398] Metallurgy is a heavy metal composition in the key of F minor built around a head- banging guitar riff that was easy toflesh out with the website.

[0399] Real-world inspiration: The sixteen-note motif was initially sketched out while playing the piano, and then later adapted into the disclosed user interface as a guitar riff.

[0400] Utilizing many operations: The composition economically develops the main riff with all of the transformations available with the website, including adding, cutting, repeating, merging, and transposing clips.

[0401] Musical sketching: Though the drum kit sounds rather artificial, we can now easily use this musical sketch to notate a full-fledged composition within a digital audio workstation.

[0402] Composition 4: Xylophonia

[0403] Xylophonia is a psychedelic composition in the keys of E dorian and A dorian featuring many layered xylophones with heavy delay effects.

[0404] Random variations: Similar to Harmonium, we were able to immediately derive many randomized 8-note melodies and variations of an original idea.

[0405] Composing with symmetry: We compose this piece with a symmetrical struc- ture, introducing or removing a pattern every four bars until the end.

[0406] Navigating polytonality: We organize our parts across two different Scale Tracks to create a polytonal piece in the keys of E dorian and A dorian.

[0407] Composition 5: Scriabinism

[0408] Scriabinism is an upbeat house composition in the key of F# major inspired by the Presto con allegrezza theme from Alexander Scriabin’s Sonata No.5. We describe this composition in great detail within Chapter 4.

[0409] Additional Information - Section E

[0410] The disclosed user interface may comprise and / or be configured to perform any one or any combination of the following commands (e.g., and may be associated with keyboard shortcuts). Under each category, we list the name of examples command (e.g., shortcuts are omitted).

[0411] General: Play / Pause, Stop, Loop / Unloop, Set Loop Start, Set Loop End, Scroll to Beat, Save File, Open File.

[0412] Timeline: Undo Timeline Change, Redo Timeline Change, Select Clip, Select Range of Clips, Select Clips Individually, Deselect Clips, Move Clips, Copy Clips, Delete Clips.514908-0584-6290.1

[0413] Pattern Editor: Show Editor, Hide Editor, Undo Pattern Change, Redo Pattern Change, Play Pattern, Add Pattern to Timeline, Export Pattern, Show / Hide Cursor, Select Note with Cursor, Move Cursor Left, Move Cursor Right, Skip Cursor Left, Skip Cursor Right, Start / Stop Adding Notes, Start / Stop Inserting Notes, Add / Insert a Note, Add a Chord, Erase Note, Clear Pattern, Add / Insert Rest, Select 16thNote, Select 8th note, Select Quarter Note, Select Whole Note, Transpose Pattern up, Transpose Pattern Down, Invert Pattern Up, Invert Pattern Down, Randomize Pattern, Contract Pattern, Expand Pattern, Slice Pattern, Repeat Pattern, Shuffle Pattern.

[0414] Scale Editor: Show Editor, Hide Editor, Undo Scale Change, Redo Scale Change, Play Scale, Export Scale, Start / Stop Adding Notes, Start / Stop Removing Notes, Add / Remove Note, Clear Scale, Transpose Scale Up, Transpose Scale Down, Invert Scale Up, Invert Scale Down.

[0415] Instrument Editor: Show Editor, Hide Editor, Play Instrument, Select Next Instrument, Select Previous Instrument, Skip to Next Category, Skip to Previous Category.

[0416] Patterns: Edit patterns, Add Pattern Clip, Cut Pattern Clip, Merge Pattern Clips, Repeat Pattern Clips, Transpose Tracks / Clips.

[0417] References

[0418] The references listed below are part of the application and are incorporated herein by reference in their entirety as if fully set forth herein for any and all purposes. 1. Opensheetmusicdisplay. https: / / opensheetmusicdisplay.org / . 2. React. https: / / react.dev / . 3. Redux. https: / / react-redux.js.org / . 4. Tone. https: / / tonejs.github.io / . 5. Typescript. https: / / www.typescriptlang.org / . 6. Adazzle. React-data-grid. https: / / github.com / adazzle / react-data-grid. 7. J. Rothstein. MIDI: A comprehensive introduction. AR Editions, Inc., 1992. 8. L. Spiegel. Manipulations of musical patterns. In Proceedings of the Symposium on Small Computers and the Arts, pages 19–22, 1981. 9. D. Tymoczko. Scale theory, serial theory and voice leading. Music Analysis, 27(1):1– 49, 2008. 10. D. Tymoczko. A Geometry of Music: Harmony and Counterpoint in the Extended Common Practice. Oxford University Press, 2010. 11. D. Tymoczko. Tonality: An owner’s manual.2023.524908-0584-6290.1

[0419] Example Computer Implementation

[0420] FIG.36 depicts a computing device that may be used in various aspects, such as the servers, modules, and / or devices depicted in FIG.1. With regard to the example architecture of FIG.1, the storage device 102, network device 104, user device 106, application device 108, output device 110, first network 112, and second network 114 may each be implemented in an instance of a computing device 3600 of FIG.36. The computer architecture shown in FIG.36 shows a conventional server computer, workstation, desktop computer, laptop, tablet, network appliance, PDA, e-reader, digital cellular phone, or other computing node, and may be utilized to execute any aspects of the computers described herein, such as to implement the methods described in relation to FIG.1, FIG.2, and / or any other method described herein.

[0421] The computing device 3600 may include a baseboard, or “motherboard,” which is a printed circuit board to which a multitude of components or devices may be connected by way of a system bus or other electrical communication paths. One or more central processing units (CPUs) 3604 may operate in conjunction with a chipset 3606. The CPU(s) 3604 may be standard programmable processors that perform arithmetic and logical operations necessary for the operation of the computing device 3600.

[0422] The CPU(s) 3604 may perform the necessary operations by transitioning from one discrete physical state to the next through the manipulation of switching elements that differentiate between and change these states. Switching elements may generally include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide an output state based on the logical combination of the states of one or more other switching elements, such as logic gates. These basic switching elements may be combined to create more complex logic circuits including registers, adders-subtractors, arithmetic logic units, floating-point units, and the like.

[0423] The CPU(s) 3604 may be augmented with or replaced by other processing units, such as GPU(s) 3605. The GPU(s) 3605 may comprise processing units specialized for but not necessarily limited to highly parallel computations, such as graphics and other visualization-related processing.

[0424] A chipset 3606 may provide an interface between the CPU(s) 3604 and the remainder of the components and devices on the baseboard. The chipset 3606 may provide an interface to a random access memory (RAM) 3608 used as the main memory in the computing device 3600. The chipset 3606 may further provide an interface to a computer-534908-0584-6290.1readable storage medium, such as a read-only memory (ROM) 3620 or non-volatile RAM (NVRAM) (not shown), for storing basic routines that may help to start up the computing device 3600 and to transfer information between the various components and devices. ROM 3620 or NVRAM may also store other software components necessary for the operation of the computing device 3600 in accordance with the aspects described herein.

[0425] The computing device 3600 may operate in a networked environment using logical connections to remote computing nodes and computer systems through local area network (LAN) 3616. The chipset 3606 may include functionality for providing network connectivity through a network interface controller (NIC) 3622, such as a gigabit Ethernet adapter. A NIC 3622 may be capable of connecting the computing device 3600 to other computing nodes over a network 3616. It should be appreciated that multiple NICs 3622 may be present in the computing device 3600, connecting the computing device to other types of networks and remote computer systems.

[0426] The computing device 3600 may be connected to a mass storage device 3628 that provides non-volatile storage for the computer. The mass storage device 3628 may store system programs, application programs, other program modules, and data, which have been described in greater detail herein. The mass storage device 3628 may be connected to the computing device 3600 through a storage controller 3624 connected to the chipset 3606. The mass storage device 3628 may comprise (e.g., or consist) of one or more physical storage units. A storage controller 3624 may interface with the physical storage units through a serial attached SCSI (SAS) interface, a serial advanced technology attachment (SATA) interface, a fiber channel (FC) interface, or other type of interface for physically connecting and transferring data between computers and physical storage units.

[0427] The computing device 3600 may store data on a mass storage device 3628 by transforming the physical state of the physical storage units to reflect the information being stored. The specific transformation of a physical state may depend on various factors and on different implementations of this description. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage units and whether the mass storage device 3628 is characterized as primary or secondary storage and the like.

[0428] For example, the computing device 3600 may store information to the mass storage device 3628 by issuing instructions through a storage controller 3624 to alter the magnetic characteristics of a particular location within a magnetic disk drive unit, the reflective or refractive characteristics of a particular location in an optical storage unit, or the544908-0584-6290.1electrical characteristics of a particular capacitor, transistor, or other discrete component in a solid-state storage unit. Other transformations of physical media are possible without departing from the scope and spirit of the present description, with the foregoing examples provided only to facilitate this description. The computing device 3600 may further read information from the mass storage device 3628 by detecting the physical states or characteristics of one or more particular locations within the physical storage units.

[0429] In addition to the mass storage device 3628 described above, the computing device 3600 may have access to other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. It should be appreciated by those skilled in the art that computer-readable storage media may be any available media that provides for the storage of non-transitory data and that may be accessed by the computing device 3600.

[0430] By way of example and not limitation, computer-readable storage media may include volatile and non-volatile, transitory computer-readable storage media and non- transitory computer-readable storage media, and removable and non-removable media implemented in any method or technology. Computer-readable storage media includes, but is not limited to, RAM, ROM, erasable programmable ROM (“EPROM”), electrically erasable programmable ROM (“EEPROM”), flash memory or other solid-state memory technology, compact disc ROM (“CD-ROM”), digital versatile disk (“DVD”), high definition DVD (“HD-DVD”), BLU-RAY, or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage, other magnetic storage devices, or any other medium that may be used to store the desired information in a non-transitory fashion.

[0431] A mass storage device, such as the mass storage device 3628 depicted in FIG.36, may store an operating system utilized to control the operation of the computing device 3600. The operating system may comprise a version of the LINUX operating system. The operating system may comprise a version of the WINDOWS SERVER operating system from the MICROSOFT Corporation. According to further aspects, the operating system may comprise a version of the UNIX operating system. Various mobile phone operating systems, such as IOS and ANDROID, may also be utilized. It should be appreciated that other operating systems may also be utilized. The mass storage device 3628 may store other system or application programs and data utilized by the computing device 3600.

[0432] The mass storage device 3628 or other computer-readable storage media may also be encoded with computer-executable instructions, which, when loaded into the computing554908-0584-6290.1device 3600, transforms the computing device from a general-purpose computing system into a special-purpose computer capable of implementing the aspects described herein. These computer-executable instructions transform the computing device 3600 by specifying how the CPU(s) 3604 transition between states, as described above. The computing device 3600 may have access to computer-readable storage media storing computer-executable instructions, which, when executed by the computing device 3600, may perform the methods described in relation to FIG.1, FIG.2, and / or any other method described herein.

[0433] A computing device, such as the computing device 3600 depicted in FIG.36, may also include an input / output controller 3632 for receiving and processing input from a number of input devices, such as a keyboard, a mouse, a touchpad, a touch screen, an electronic stylus, or other type of input device. Similarly, an input / output controller 3632 may provide output to a display, such as a computer monitor, a flat-panel display, a digital projector, a printer, a plotter, or other type of output device. It will be appreciated that the computing device 3600 may not include all of the components shown in FIG.36, may include other components that are not explicitly shown in FIG.36, or may utilize an architecture completely different than that shown in FIG.36.

[0434] As described herein, a computing device may be a physical computing device, such as the computing device 3600 of FIG.36. A computing node may also include a virtual machine host process and one or more virtual machine instances. Computer-executable instructions may be executed by the physical hardware of a computing device indirectly through interpretation and / or execution of instructions stored and executed in the context of a virtual machine.

[0435] It is to be understood that the methods and systems are not limited to specific methods, specific components, or to particular implementations. It is also to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

[0436] As used in the specification and the appended claims, the singular forms “a,” “an,” and “the” include plural referents unless the context clearly dictates otherwise. Ranges may be expressed herein as from “about” one particular value, and / or to “about” another particular value. When such a range is expressed, another embodiment includes from the one particular value and / or to the other particular value. Similarly, when values are expressed as approximations, by use of the antecedent “about,” it will be understood that the particular value forms another embodiment. It will be further understood that the endpoints of each of564908-0584-6290.1the ranges are significant both in relation to the other endpoint, and independently of the other endpoint.

[0437] “Optional” or “optionally” means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0438] Throughout the description and claims of this specification, the word “comprise” and variations of the word, such as “comprising” and “comprises,” means “including but not limited to,” and is not intended to exclude, for example, other components, integers or steps. “Exemplary” means “an example of” and is not intended to convey an indication of a preferred or ideal embodiment. “Such as” is not used in a restrictive sense, but for explanatory purposes.

[0439] The term “or” when used with “one or more of” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some or all of the elements in the list. The term “or” when used with “at least one of” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some or all of the elements in the list. For example, the phrases “one or more of A, B, or C” includes any of the following: A, B, C, A and B, A and C, B and C, and A and B and C. Similarly the phrase “one or more of A, B, and C” includes any of the following: A, B, C, A and B, A and C, B and C, and A and B and C. The phrase “at least one of A, B, or C” includes any of following: A, B, C, A and B, A and C, B and C, and A and B and C. Similarly, the phrase “at least one of A, B, and C” includes any of following: A, B, C, A and B, A and C, B and C, and A and B and C.

[0440] Components are described that may be used to perform the described methods and systems. When combinations, subsets, interactions, groups, etc., of these components are described, it is understood that while specific references to each of the various individual and collective combinations and permutations of these may not be explicitly described, each is specifically contemplated and described herein, for all methods and systems. This applies to all aspects of this application including, but not limited to, operations in described methods. Thus, if there are a variety of additional operations that may be performed it is understood that each of these additional operations may be performed with any specific embodiment or combination of embodiments of the described methods.574908-0584-6290.1

[0441] As will be appreciated by one skilled in the art, the methods and systems may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the methods and systems may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. More particularly, the present methods and systems may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be utilized including hard disks, CD-ROMs, optical storage devices, or magnetic storage devices.

[0442] Embodiments of the methods and systems are described herein with reference to block diagrams and flowchart illustrations of methods, systems, apparatuses and computer program products. It will be understood that each block of the block diagrams and flowchart illustrations, and combinations of blocks in the block diagrams and flowchart illustrations, respectively, may be implemented by computer program instructions. These computer program instructions may be loaded on a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions which execute on the computer or other programmable data processing apparatus create a means for implementing the functions specified in the flowchart block or blocks.

[0443] These computer program instructions may also be stored in a computer-readable memory that may direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including computer-readable instructions for implementing the function specified in the flowchart block or blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions that execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks.

[0444] The various features and processes described above may be used independently of one another, or may be combined in various ways. All possible combinations and sub- combinations are intended to fall within the scope of this disclosure. In addition, certain methods or process blocks may be omitted in some implementations. The methods and584908-0584-6290.1processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto may be performed in other sequences that are appropriate. For example, described blocks or states may be performed in an order other than that specifically described, or multiple blocks or states may be combined in a single block or state. The example blocks or states may be performed in serial, in parallel, or in some other manner. Blocks or states may be added to or removed from the described example embodiments. The example systems and components described herein may be configured differently than described. For example, elements may be added to, removed from, or rearranged compared to the described example embodiments.

[0445] It will also be appreciated that various items are illustrated as being stored in memory or on storage while being used, and that these items or portions thereof may be transferred between memory and other storage devices for purposes of memory management and data integrity. Alternatively, in other embodiments, some or all of the software modules and / or systems may execute in memory on another device and communicate with the illustrated computing systems via inter-computer communication. Furthermore, in some embodiments, some or all of the systems and / or modules may be implemented or provided in other ways, such as at least partially in firmware and / or hardware, including, but not limited to, one or more application-specific integrated circuits (“ASICs”), standard integrated circuits, controllers (e.g., by executing appropriate instructions, and including microcontrollers and / or embedded controllers), field-programmable gate arrays (“FPGAs”), complex programmable logic devices (“CPLDs”), etc. Some or all of the modules, systems, and data structures may also be stored (e.g., as software instructions or structured data) on a computer-readable medium, such as a hard disk, a memory, a network, or a portable media article to be read by an appropriate device or via an appropriate connection. The systems, modules, and data structures may also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagated signal) on a variety of computer-readable transmission media, including wireless-based and wired / cable-based media, and may take a variety of forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). Such computer program products may also take other forms in other embodiments. Accordingly, the present invention may be practiced with other computer system configurations.

[0446] While the methods and systems have been described in connection with preferred embodiments and specific examples, it is not intended that the scope be limited to the594908-0584-6290.1particular embodiments set forth, as the embodiments herein are intended in all respects to be illustrative rather than restrictive.

[0447] It will be apparent to those skilled in the art that various modifications and variations may be made without departing from the scope or spirit of the present disclosure. Other embodiments will be apparent to those skilled in the art from consideration of the specification and practices described herein. It is intended that the specification and example figures be considered as exemplary only, with a true scope and spirit being indicated by the following claims.

[0448] Aspects

[0449] The disclosed techniques may include any combination of the following aspects. These aspects may be combined in any way with each other and / or with any features disclosed further elsewhere herein.

[0450] Aspect 1. A method comprising, consisting of, or consisting essentially of: causing a user interface to output a first interface object for visually representing a first musical property in a musical composition; determining, based on user input via the user interface, a hierarchical relationship between the first interface object and a second interface object, wherein the hierarchical relationship indicates inheritance of the first musical property from the first interface object to the second interface object such that a change made to the first musical property in the first interface object causes an update to the second interface object; and causing output, via the user interface, of a representation of the hierarchical relationship.

[0451] Aspect 2. The method of Aspect 1, wherein the first interface object is a different type of interface object than the second interface object, wherein the second interface object is associated with a second musical property different from the first musical property associated with the first interface object.

[0452] Aspect 3. The method of any one of Aspects 1-2, wherein the first interface object comprises a scale object indicating a musical scale to apply to one or more descendent objects via inheritance from the first interface object, and wherein the first musical property comprises an indication of one of a plurality of available musical scales (e.g., or collections).

[0453] Aspect 4. The method of Aspect 3, wherein the first interface object allows a user to edit the musical scale independent from musical notes associated with the second interface object.604908-0584-6290.1

[0454] Aspect 5. The method of any one of Aspects 1-4, wherein the second interface object comprises a pattern object indicating a sequence of musical notes, wherein the pattern object indicates an instrument associated with the sequence of musical notes.

[0455] Aspect 6. The method of Aspect 5, wherein the second interface object allows a user to edit the sequence of musical notes.

[0456] Aspect 7. The method of any one of Aspects 1-6, wherein the user input comprises a user indicating a type of object of the second interface object and a relationship between the first interface object and the second interface object.

[0457] Aspect 8. The method of any one of Aspects 1-7, further comprising determining, based on user input, a pose object configured to apply an operation that modifies the second interface object for a time range.

[0458] Aspect 9. The method of Aspect 8, wherein the operation comprises one or more of transposition of notes, changing a scale, modifying a note, modifying musical properties of descendent interface objects, generating a voice leading, changing a set of musical notes to another set of musical notes, or changing a set of pitches to another set of pitches.

[0459] Aspect 10. The method of any one of Aspects 1-9, wherein the user interface comprises a timeline view showing the one or more interface objects along a timeline.

[0460] Aspect 11. The method of any one of Aspects 1-10, wherein the user interface comprises a tree view that represents the one or more interface objects as a hierarchical tree graph, wherein a respective node is shown for each corresponding interface object and a line between two nodes of the respective nodes represents a hierarchical relationship between the two nodes.

[0461] Aspect 12. The method of any one of Aspects 1-11, wherein the user interface is configured to allow a user to edit the hierarchical relationship.

[0462] Aspect 13. The method of any one of Aspects 1-12, wherein the user interface is configured to allow a user to define a transposition operation comprising one or more of a chromatic modification, a scalar modification, or a chordal modification.

[0463] Aspect 14. The method of any one of Aspects 1-13, wherein the user interface is configured to allow a user to modify multiple objects in a hierarchy of interface objects to define musical properties inherited by multiple descendants of an interface object.

[0464] Aspect 15. The method of any one of Aspects 1-14, wherein causing output comprises one or more of updating the user interface, causing storage of the hierarchical614908-0584-6290.1relationship, or sending via a network data indicative of the hierarchical relationship to a computing device outputting the user interface.

[0465] Aspect 16. The method of any one of Aspects 1-15, wherein updating the first musical property causes a plurality of updates to a plurality of interface objects inheriting via corresponding hierarchical relationships from the first interface object.

[0466] Aspect 17. The method of any one of Aspects 1-16, wherein the user interface comprises one or more of a digital audio workstation or a notation program.

[0467] Aspect 18. The method of any one of Aspects 1-17, wherein the second interface object comprises a midi track.

[0468] Aspect 19. The method of any one of Aspects 1-18, wherein determining the hierarchical relationship comprises processing one or more of an audio file or a representation of the musical composition and generating, based on the processing, data indicative of a hierarchical relationship, wherein processing comprises determining a scale object indicating a musical scale, determining a pattern object indicating a series of musical notes, and determining the hierarchical relationship between the scale object and the pattern object.

[0469] Aspect 20. The method of any one of Aspects 1-19, further comprising causing output of the musical composition by one or more of storing data indicative of the musical composition or causing a speaker to output an audio performance of the musical composition.

[0470] Aspect 21. A method for encoding audio, comprising: implementing a hierarchical organization of musical scales by allowing one or more tracks to contain a reference to a Pattern Track and a Scale Track, where the scale track defines the scale for the patterns played by its associated parent track.

[0471] Aspect 22. The method of Aspect 21, wherein where Scale Tracks are nested Scale Tracks corresponding to musical scales, and Pattern Tracks are instrumental Pattern Tracks that serialize scale-based patterns into MIDI notes.

[0472] Aspect 23. The method of claim 22, wherein each Scale Track contains a dedicated editor for customizing its corresponding scale, which is defined using scale degrees relative to the Pattern Track’s scale or the chromatic scale if at the highest level.

[0473] Aspect 24. The method of any one of Aspects 21-23, further comprising adding one or more vectoral transpositions, each vectoral transposition added to a specific beat in a Scale Track or Pattern Track.624908-0584-6290.1

[0474] Aspect 25. The method of Aspect 24, wherein each vectoral transposition is labelled as Nx • T(X1, X…, Xn) • tx, where Nx is an offset along a chromatic scale, T(X1, X…, Xn) are offsets along one or more parent scales, and tx is an offset along an intrinsic scale.

[0475] Aspect 26. A method for decoding audio, comprising: pre-processing every media element in an audio file containing one or more tracks to contain a reference to a Pattern Track and a Scale Track and optionally one or more vectoral transpositions, where the scale track defines the scale for the patterns played by its associated parent track; rendering any spatiotemporal portals and chunking the pattern track and scale track into fragments accordingly; and arranging the fragments based on an entrance time and / or tick and exit time and / or tick of each spatiotemporal portals.

[0476] Aspect 27. The method of Aspect 26, wherein where Scale Tracks are nested Scale Tracks corresponding to musical scales, and Pattern Tracks are instrumental Pattern Tracks that serialize scale-based patterns into MIDI notes.

[0477] Aspect 28. The method of Aspect 27, wherein each Scale Track contains a dedicated editor for customizing its corresponding scale, which is defined using scale degrees relative to the Pattern Track’s scale or the chromatic scale if at the highest level.

[0478] Aspect 29. The method of any one of Aspects 26-28, further comprising adding one or more vectoral transpositions, each vectoral transposition added to a specific beat in a Scale Track or Pattern Track.

[0479] Aspect 30. The method of Aspect 29, wherein each vectoral transposition is labelled as Nx• T(X1, X…, Xn) • tx, where Nxis an offset along a chromatic scale, T(X1, X…, Xn) are offsets along one or more parent scales, and txis an offset along an intrinsic scale.

[0480] Aspect 31. The method of any one of Aspects 26-30, further comprising adding a spatiotemporal portal to indicate an entrance and exit of a portal that can directly tunnel media from one track and / or tick to another track and / or tick.

[0481] Aspect 32. A system, comprising: one or more processing units; and a non- transitory computer readable storage device containing instructions that, when executed by the one or more processing units, causes the one or more processing units to, collectively: be configured to receive one or more pattern tracks and / or one or more scale tracks; encode audio according to a method of Aspect 21; digitally store audio encoded according to the method of Aspect 21; decode audio according to a method of Aspect 21; and provide a graphical user interface configured to display information to a user about the one or more634908-0584-6290.1Pattern Tracks and / or one or more Scale Tracks; and receive information from a user relating to how audio should be encoded.

[0482] Aspect 33. A device comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the device to perform the methods of any one of Aspects 1-31.

[0483] Aspect 34. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a device to perform the methods of any one of Aspects 1-31.

[0484] Aspect 35. A system comprising: a first computing device configured to store data associated with providing a user interface for editing an audio composition; and a second computing device comprising one or more processors, and a memory, wherein the memory stores instructions that, when executed by the one or more processors, cause the second computing device to perform the methods of any one of Aspects 1-31.644908-0584-6290.1

Claims

CLAIMS 1. A method comprising: causing a user interface to output a first interface object for visually representing a first musical property in a musical composition; determining, based on user input via the user interface, a hierarchical relationship between the first interface object and a second interface object, wherein the hierarchical relationship indicates inheritance of the first musical property from the first interface object to the second interface object such that a change made to the first musical property in the first interface object causes an update to the second interface object; and causing output, via the user interface, of a representation of the hierarchical relationship.

2. The method of claim 1, wherein the first interface object is a different type of interface object than the second interface object, wherein the second interface object is associated with a second musical property different from the first musical property associated with the first interface object.

3. The method of any one of claims 1-2, wherein the first interface object comprises a scale object indicating a musical scale or collection to apply to one or more descendent objects via inheritance from the first interface object, and wherein the first musical property comprises an indication of one of a plurality of available musical scales.

4. The method of claim 3, wherein the first interface object allows a user to edit the musical scale independent from musical notes associated with the second interface object.

5. The method of any one of claims 1-4, wherein the second interface object comprises a pattern object indicating a sequence of musical notes, wherein the pattern object indicates an instrument associated with the sequence of musical notes.

6. The method of claim 5, wherein the second interface object allows a user to edit the sequence of musical notes.654908-0584-6290.

17. The method of any one of claims 1-6, wherein the user input comprises a user indicating a type of object of the second interface object and a relationship between the first interface object and the second interface object.

8. The method of any one of claims 1-7, further comprising determining, based on user input, a pose object configured to apply an operation that modifies the second interface object for a time range.

9. The method of claim 8, wherein the operation comprises one or more of transposition of notes, changing a scale, modifying a note, modifying musical properties of descendent interface objects, generating a voice leading, changing a set of musical notes to another set of musical notes, or changing a set of pitches to another set of pitches.

10. The method of any one of claims 1-9, wherein the user interface comprises a timeline view showing the one or more interface objects along a timeline.

11. The method of any one of claims 1-10, wherein the user interface comprises a tree view that represents the one or more interface objects as a hierarchical tree graph, wherein a respective node is shown for each corresponding interface object and a line between two nodes of the respective nodes represents a hierarchical relationship between the two nodes.

12. The method of any one of claims 1-11, wherein the user interface is configured to allow a user to edit the hierarchical relationship.

13. The method of any one of claims 1-12, wherein the user interface is configured to allow a user to define a transposition operation comprising one or more of a chromatic modification, a scalar modification, or a chordal modification.

14. The method of any one of claims 1-13, wherein the user interface is configured to allow a user to modify multiple objects in a hierarchy of interface objects to define musical properties inherited by multiple descendants of an interface object.664908-0584-6290.

115. The method of any one of claims 1-14, wherein causing output comprises one or more of updating the user interface, causing storage of the hierarchical relationship, or sending via a network data indicative of the hierarchical relationship to a computing device outputting the user interface.

16. The method of any one of claims 1-15, wherein updating the first musical property causes a plurality of updates to a plurality of interface objects inheriting via corresponding hierarchical relationships from the first interface object.

17. The method of any one of claims 1-16, wherein the user interface comprises one or more of a digital audio workstation or a notation program.

18. The method of any one of claims 1-17, wherein the second interface object comprises a midi track.

19. The method of any one of claims 1-18, wherein determining the hierarchical relationship comprises processing one or more of an audio file or a representation of the musical composition and generating, based on the processing, data indicative of a hierarchical relationship, wherein processing comprises determining a scale object indicating a musical scale, determining a pattern object indicating a series of musical notes, and determining the hierarchical relationship between the scale object and the pattern object.

20. The method of any one of claims 1-19, further comprising causing output of the musical composition by one or more of storing data indicative of the musical composition or causing a speaker to output an audio performance of the musical composition.

21. A device comprising: one or more processors; and a memory storing instructions that, when executed by the one or more processors, cause the device to perform the methods of any one of claims 1-20.

22. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause a device to perform the methods of any one of claims 1-20.674908-0584-6290.

123. A system comprising: a first computing device configured to store data associated with providing a user interface for editing an audio composition; and a second computing device comprising one or more processors, and a memory, wherein the memory stores instructions that, when executed by the one or more processors, cause the second computing device to perform the methods of any one of claims 1-20.684908-0584-6290.1

Images