Audio composition and playback

Abstract

A method comprising deriving a reduced set of one or more note groups based at least in part on a recorded set of note groups each having one or more notes, each recorded set of note groups having one or more recorded audio properties, identifying a terminating condition of a terminal note in the reduced set, recording a first audio file responsive to identifying that the terminal note exhibits a first predetermined terminating condition, the first audio file being recorded with a first temporal property and capturing a first note group of the reduced set, and recording a second audio file responsive to identifying that the terminal note exhibits a second predetermined terminating condition, the second audio file being recorded with a second temporal property different from the first temporal property and capturing a second note group of the reduced set.

Description

AUDIO COMPOSITION AND PLAYBACKBACKGROUND

[0001] The process of learning to play a typical musical instrument is time-consuming and poses a high barrier to entry that often limits a player’s proficiency and engagement with the instrument. Among various challenges that contribute to this barrier are the development of proper and proficient technique, which often takes many hours to achieve, understanding the complexities of harmony (e.g., intervals, scales, modes), and achieving a desirable sound through the selection and physical configuration of an instrument and associated equipment such as effects processors and amplifiers. In addition to dissuading a player from playing an instrument, these and other challenges limit players’ abilities to compose music, share compositions, and play with other musicians. Further, the physical demands imposed by an instrument can be restrictive or prohibitive for players with physical disabilities.SUMMARY

[0002] Examples are provided that relate to music composition and playback via audio files. One example provides a method comprising: deriving a reduced set of one or more note groups based at least in part on a recorded set of note groups each having one or more notes, each recorded set of note groups having one or more recorded audio properties; identifying a terminating condition of a terminal note in the reduced set; recording a first audio file responsive to identifying that the terminal note exhibits a first predetermined terminating condition, the first audio file being recorded with a first temporal property and capturing a first note group of the reduced set; and recording a second audio file responsive to identifying that the terminal note exhibits a second predetermined terminating condition, the second audio file being recorded with a second temporal property different from the first temporal property and capturing a second note group of the reduced set. The method further comprises: creating a packet file that includes a corresponding audio file for each note group in the reduced set, the packet file including at least the first and second audio files, each corresponding audio file being recorded with one or more packet audio properties selected based at least on the one ormore recorded audio properties; receiving a first user input that identifies a selected one or more of the audio files in the packet file; inserting the selected one or more audio files into a playline displayed on an interface; receiving a second user input to cause playback of the selected one or more audio files inserted into the playline; and receiving a third user input to cause creation of a file in a file format that identifies a sequential order for playback of the selected one or more audio files inserted into the playline and does not identify a timing at which to play back the selected one or more audio files.

[0003] This Summary is provided to introduce in a simplified form a selection of concepts that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.BRIEF DESCRIPTION OF THE DRAWINGS

[0004] Fig. 1 depicts an example audio system configured to record audio files for composition and playback.

[0005] Fig. 2 depicts an example musical instrument configured for audio file playback.

[0006] Fig. 3 depicts a user interface of an example application for creating compositions from audio files.

[0007] Fig. 4 schematically depicts an example system for recording audio files, generating packet files that comprise audio files, and creating playback files that define a sequence for playback of audio files.

[0008] Fig. 5 depicts a flowchart illustrating an example method of recording audio files.

[0009] Fig. 6 depicts a flowchart illustrating an example method of creating a file for playback of audio files.

[0010] Fig. 7 schematically depicts an example computing system.DETAILED DESCRIPTION

[0011] As mentioned above, the process of learning to play a typical musical instrument is time-consuming and poses a high barrier to entry that often limits a player’s proficiency and engagement with the instrument. Among various challenges that contribute to this barrier are the development of proper technique, understanding the complexities of music theory and harmony (e.g., intervals, scales, modes), learning to read music (e.g., staff notation), and achieving a desirable sound through the selection and physical configuration of an instrument and associated equipment such as effects processors and amplifiers. In addition to dissuading a player from playing an instrument, these and other challenges limit players’ abilities to compose music, share compositions, and play with other musicians. Further, the physical demands imposed by an instrument can be restrictive or prohibitive for players with physical disabilities such as visual or motor-related disabilities.

[0012] To address these and other issues associated with typical musical instruments and the traditional experience of playing music, examples are disclosed that enable music to be composed and played though the creation and playback of audio files that each capture a recording of a group of one or more notes. Application interfaces and systems described below provide user interface-based selection and sequencing of note groups to enable note-group sequencing and music composition. Audio files that capture a note group can be mapped to an instrument button or other input mechanism for playback at a timing based on engagement with the mechanism. Note groups recorded with selected properties — e.g., capturing selected intervals, rhythms, and / or instrument-related articulations such as hammer-ons and pull-offs on a guitar — can comprise a relatively small set of audio files that when combined and sequenced provide harmonic and rhythmic variety. In some examples, to obtain an expanded set of note groups from a relatively smaller set of recorded note groups, audio processing techniques, such as pitch-shifting to obtain a greater number of intervals, and processing that enables a note group having one duration to be generated from a note group previously recorded with a different duration, can be used.

[0013] Examples are also disclosed in which audio files are recorded that target an identified performance or sound. For example, a set of note groups can be recorded withselected properties such as intervals, rhythms, and / or articulations that when played back in a defined sequence result in the performance of a targeted part in a recording, such as a solo or rhythm part of a song. In some such examples, the set of note groups can be recorded with audio properties that in relation to an instrument on which the targeted part was performed, target a sound such as a tone or timbre exhibited by the instrument. Further examples are possible in which, in addition to recording note groups with audio properties that target a sound exhibited in a targeted part, note groups are recorded with intervals, rhythms, and / or articulations not exhibited in the targeted part, enabling players to compose parts with their chosen musical content in the targeted sound and style.

[0014] The creative agency afforded by the disclosed approaches can be comparable to or exceed that provided by a typical musical instrument, including instruments used by musicians having knowledge of music theory and proficiency on such instruments. However, this creative agency is afforded without requiring knowledge of music theory or proficiency on a typical musical instrument. Instead, music composition and performance using the disclosed techniques can be achieved in a relatively more intuitive manner using one’s senses as guides.

[0015] Fig. 1 depicts an example audio system 100 configured to record audio files for composition and playback. In the depicted example, audio files recorded via audio system 100 target a solo part, performed by an instrument captured in audio 102, that is received from an audio source 104 communicatively coupled to the audio system. The solo part is analyzed to identify note groups — and intervals, rhythms, and / or articulations in identified note groups — that comprise the solo part. Audio system 100 can be used to record audio files that capture note groups having intervals, rhythms, and / or articulations based on those in the identified note groups of audio 102, and potentially with audio properties that target a sound exhibited by an instrument that played the solo part. As described below, audio files recorded via audio system 100 can be played back in a defined sequence at timings determined by user input to achieve a performance of the targeted solo part in audio 102.

[0016] Fig. 1 graphically depicts, in the form of staff notation, example note groups captured in audio 102 and analyzed via audio system 100. However, it will be understood that the depicted staff representation of audio 102 is presented for the purpose of illustration, andin this example is not rendered by audio system 100 for display. While in other examples a staff or other graphical representation of audio 102 may be rendered by audio system 100, the depicted example contemplates use of the audio system to perform acoustic analysis of the waveform of the audio to identify note groups captured therein.

[0017] Analysis of audio 102 via audio system 100 identifies note groups captured in the audio that include various intervals and rhythms. Among the identified note groups is a note group 106A comprising one note (a dotted quarter note), a note group 106B comprising two notes (two eighth notes), a note group 106C comprising one note (a half note), a note group 106D comprising one note (a dotted quarter note), and a note group 106E comprising three notes (an eighth-note triplet). However, analysis of identified note groups 106 indicates that note group 106A and note group 106D exhibit a common rhythm (dotted quarter note) and common interval. Other note groups that exhibit common interval (s), rhythm(s), and / or articulation(s) can also be identified. As such, a reduced set 108 of note groups (e.g., note group 110) that comprise the solo part analyzed in audio 102, and that can be combined in a defined sequence to achieve a performance of the solo part, can be derived from the relatively greater number of note groups 106 initially identified in the solo part. In this way, audio system 100 can be used to identify the set of note groups in a given part of an audio recording that are representative or reconstructive of the harmonic and rhythmic content of the part and that can be used to reproduce the part, while discarding note groups that are duplicative or otherwise overlap in harmonic and / or rhythmic content. In some examples, this reduction process may identify and produce a reduced set of unique note groups (e.g., note groups that differ in one or more of intervals, rhythms, and articulations).

[0018] Note groups 106, including intervals, rhythms, and instrument-related articulations exhibited by such note groups, can be identified in audio 102 in any suitable manner. In some examples, audio system 100 can implement pitch detection, waveform analysis (e.g., analyzing waveform amplitude over time, spectral decomposition or other frequency analysis), and / or any other suitable techniques to identify note groups 106 and harmonic and rhythmic content therein. For example, audio system 100 can analyze the amplitude of a note group 106 over time to identify a rhythm exhibited by the note group and durations of notes in the note group,which can be identified to determine how the note group is terminated — e.g., whether the note group comes to an abrupt stop or is held for a length of time. In some examples, user input provided to audio system 100 can identify, or assist in the identification of, note groups 106 and associated harmonic and / or rhythmic content. As one example, user input may be received via a graphical user interface that confirms the identification of individual note groups 106.

[0019] Continuing with the example depicted in Fig. 1, audio system 100 is used to record a corresponding audio file 112 for each note group 110 in the reduced set 108 of note groups. Each audio file 112 captures a recording of a corresponding note group 110 in the reduced set 108, including the intervals, rhythms, and / or articulations exhibited by the corresponding note group. Any suitable method of recording audio files 112 can be employed. In some examples, a live performance of a note group 110, including its harmonic and rhythmic content, can be recorded by audio system 100. The performance can be carried out on an instrument that played the targeted solo part in audio 102, for example. In other examples, audio system 100 can record a performance of a note group 110 carried out by a synthesizer or other programmable device.

[0020] The recording process can include targeting a sound exhibited by the solo part in audio 102 from which note groups 106 are derived. As an example in which the targeted solo part in audio 102 is a guitar solo, the sound of a guitar (or its output having passed through a signal chain comprising effects, processing, and / or amplification) used to record a performance of note groups 110 in creating audio files 112 can be configured to target the guitar solo in audio 102, such as through the selection of the guitar and its physical configuration, the use and settings of effects processors (e.g., reverb, delay, gain, modulation) and amplification, and the use and settings of audio processing (e.g., compression, equalization, noise suppression).

[0021] Audio files 112 recorded via audio system 100 can be played back to achieve a performance of the solo part targeted in audio 102. Fig. 1 depicts an example in which audio files 112, which capture note groups 110 in the reduced set 108, are identified (e.g., through filenames of each audio file) in a playback file 114 that defines a sequence for playback of the identified audio files. When played back in this defined sequence, the audio files enable the performance of the targeted solo in the previously analyzed audio 102. As an example, Fig. 1depicts a digital musical instrument 116, configured with a form factor of a guitar, with which audio files 112 can be played back in the sequence defined by playback file 114 at timings effected by user input provided to the instrument. However, any suitable instrument can be used to play back audio files 112 at timings effected by user input with any suitable form factors, including but not limited to that of a drum kit or other percussive instrument, saxophone or other wind instrument, and other stringed instruments.

[0022] Fig. 2 depicts an example musical instrument 200 configured with a guitar form factor. Instrument 200 includes an input device implemented as a touchscreen 202, which is disposed along the neck of the instrument. Touchscreen 202 is operable to display a plurality of virtual buttons (e.g., virtual button 204) that when engaged by a user (e.g., through touch input applied to the touchscreen) cause playback of an audio file capturing a recording of a note group, such as an audio file 112. Instrument 200 further includes a storage device 206 on which a plurality of audio files, along with a playback file (e.g., playback file 114) that identifies a sequence for playback of the plurality of audio files, can be stored. In such examples, the playback file can map each audio file to a virtual button 204 on instrument 200 such that a user can engage the virtual buttons in the sequence defined by the playback file to cause playback of the audio files in that sequence and achieve a performance of the note groups captured in the audio files. While in this example the sequence for playback is defined by the playback file, the timing at which each audio file is played back is controlled by the user — each audio file is played back in response to the engagement of a virtual button 204 to which that audio file is mapped. In this way, a balance can be achieved between enabling music performance, without the demands of music theory knowledge and traditional instrument proficiency, with user agency through timing playback based on user input.

[0023] Touchscreen 202 can display indications of the next virtual button 204 to be engaged for playing back the next audio file in the sequence identified by the playback file. As one example, Fig. 2 depicts a virtual button 204 displayed with shading, indicating that this virtual button is the next button to be engaged to cause playback of the next audio file in the sequence. Alternatively or additionally, touchscreen 202 can display indications of a currently engaged virtual button 204. Further, other types of indications (of a button being currentlyengaged, next button to engage in a sequence, and / or other indications) and feedback can be provided to a user of instrument 200, such as thermal feedback and / or haptic feedback provided at touchscreen 202, which can provide a thermal and / or tactile sensation localized to the area in which a virtual button 204 is displayed to indicate current user engagement with the button and / or a future button to be engaged.

[0024] Touchscreen 202 can implement any suitable touch-sensing technologies, including but not limited to capacitive, resistive, and optical touch sensors. In yet other examples, touchscreen 202 can be configured to detect hover input applied by a user and / or a measurement of applied force. Further, implementations are contemplated in which physical buttons are provided alternatively or in addition to virtual buttons 204. In such implementations, a physical button can be depressed to cause playback of a mapped audio file, and can potentially be configured to display or provide other feedback regarding a currently depressed button and / or future button to be depressed.

[0025] Any suitable mapping between note groups captured in audio files and virtual buttons 204 can be established. For example, one or more buttons 204 can be provided for note groups that include a single note, where notes of different intervals are mapped to different buttons. In some examples, this can include a one-to-one correspondence, such that each button 204 is associated with one interval. In other examples, multiple intervals can be mapped to a button 204. Further, button 204 can be provided for a single octave, or in other examples buttons can be provided for multiple octaves. Still further, buttons 204 can be provided for note groups that include chords. Still further, buttons 204 can be provided for one or more instrument-related articulations — as an example in which guitar-related articulations are recorded in audio fdes, a respective button can be provided for each of an audio file capturing a hammer-on, an audio file capturing a slide, an audio file capturing a pull-off, and an audio file exhibiting a bend. In other examples, multiple articulations can be mapped to a button 204. In yet other examples, one or more buttons 204 can be configured as an expression control that activates or deactivates one or more articulations upon engagement of the button(s). Instrument 200 can implement any suitable combination of the aforementioned mappings, and / or any other suitable type of mapping. As a particular example in the context of a guitar,one exemplary mapping that can be implemented in instrument 200 includes: a button 204 mapped to each note in a scale (e.g., a seven-note major scale each note of which is mapped to a corresponding button of a set of seven scale buttons), a button mapped to each chord in a set of chords, and a button mapped to each articulation in a set of articulations (e.g., a respective button mapped to each of a slide, pull-off, hammer-on, and bend).

[0026] In some examples, instrument 200 can include a strum area 208 that implements a sensor system to detect user input provided at or proximate to the strum area. In one implementation, strum area 208 can include a magnetic sensor configured to detect the presence of a magnetic element, such as a magnetic plectrum. Strumming performed by a user of instrument 200 with the magnetic plectrum can be detected at strum area 208, and potentially used in combination with detecting the engagement of virtual buttons 204, to cause audio file playback. For example, audio file playback can be effected in response to detecting engagement of a button 204 to which the audio file is mapped, in addition to detecting strumming at strum area 208 within a threshold duration of the time at which the button is engaged. Where strum area 208 is configured with a magnetic element (e.g., the magnetic sensor itself or another element), magnetic interaction between the magnetic element in the strum area and a magnetic plectrum can provide a locomotive sensation of strumming a guitar, yet without configuring instrument 200 with physical strings. However, implementations are contemplated in which strum area 208 can be engaged through touch and / or hover input (e.g., detected via a touch sensor), actuating input (e.g., applied to a physical actuator), and / or any other suitable type of input.

[0027] Various forms of audio file playback at instrument 200 are contemplated. Fig. 2 illustrates examples in which instrument 200 includes a speaker 210 configured to play back audio files prompted by user input as described herein. In this example, speaker 210 is integrated in the body of instrument 200. Alternatively or additionally, instrument 200 can include an output interface 212 configured to provide output signals that capture audio files. Output interface 212 can include a digital and / or analog interface (e.g., ’A" audio jack interface, stereo mini jack interface for output to headphones). As described above, the recording of audio files and note groups captured therein can include the application of effects, equalization,amplification, modeling, and / or other processing that results in a mastered sound that can be provided directly as input into a speaker, amplifier, or other output device, and without other audio processing to achieve a desirable, mastered sound.

[0028] Instrument 200 can further include a data interface 214 configured to receive audio files and playback files that identify a sequence for playback of a set of audio files. In some examples, an application described below can be used to upload a set of audio files, and playback files for the set of audio files, to instrument 200 via data interface 214 for storage at storage device 206. Audio files and playback files uploaded to instrument 200 can then be retrieved from storage device 206 for playback. In some examples, a set of audio files — which may be bundled together in what can be referred to as a packet file — can be encrypted on a per-instrument basis. In such examples, a decryption key can be provided for an identified instrument that decrypts and unlocks the packet file for playback at that instrument and not other instruments.

[0029] Instrument 200 can include other elements not depicted in Fig. 2. For example, virtual and / or physical buttons can be provided to enable bookmarking functionality with which a user can access identified positions in a playback file, potentially enabling the user to practice sections in a sequence or part. Further, instrument 200 can include a power interface for receiving external power and / or an integrated power source (e g., rechargeable battery).

[0030] Fig. 3 depicts a user interface 300 of an example application 302 that can be used to create compositions from audio files and produce playback files that identify sequences of audio files for playback (e.g., at instrument 200). Application 302 can be executed at or otherwise provided by audio system 100, for example.

[0031] User interface 300 displays a plurality of icons 304 that each represent a corresponding audio file and note group, captured in the audio file, that exhibits interval(s), rhythm(s), and / or articulations. In the depicted example, note groups for which audio files are provided comprise single-note groups, two- and three-note groups, note groups that include a chord, and note groups that exhibit articulations such as harmonics, bends, and slides. Collectively, the displayed note groups provide a variety of harmonic and rhythmic content with which to compose. The note groups can be derived in any suitable manner. For example,the displayed note groups can form a reduced set of note groups derived from a part in a song (e.g., solo or rhythm part of an identified instrument), where the reduced set collectively provides the harmonic and rhythmic content to reproduce the part in a performance through the playback of audio files that capture the reduced set. In some examples, the displayed note groups can include, in addition to the reduced set, note groups not derived from the song part, enabling the composition of harmonic and rhythmic content not found in the part. However, such note groups not derived from the song part can be recorded with a sound targeting the sound exhibited by the song part, which can enable users of application 302 to compose with an expanded set of harmonic and rhythmic content in the style and sound of the targeted part. Further, examples are contemplated in which a set of audio files are provided with harmonic content in a common key.

[0032] Each icon 304 is selectable to play back the represented audio file. This can enable a user of application 302 to choose which audio files are included in a composition, as guided by listening. Further, each icon 304 is selectable for insertion into a playline 306 displayed on user interface 300. Playline 306 establishes a temporal sequence for playback of audio files inserted into the playline. As one example, inserted audio files can be played back from left to right and top to bottom in playline 306. In this way, compositions can be created by inserting audio files into playline 306 via corresponding icons 304 (e.g., by dragging and dropping).

[0033] The sequence for playback of audio files established via playline 306 can be saved to a playback file (e.g., playback file 114) that identifies the sequence. This playback file, along with the audio files identified by the playback file, can be uploaded to an instrument for playback in the order identified by the sequence. However, the playback file may not identify a timing at which to play back audio files (or a tempo or other timing-related information other than the sequence). In such examples, the timing at which each audio file is played back can be determined by user engagement with the instrument. As one example, the playback file and identified audio files can be uploaded to instrument 200, through data interface 214, for storage at storage device 206. The playback file can then be retrieved from storage device 206, with each audio file in the packet file being played back according to its position in the identified sequence upon user engagement of a corresponding button 204 to which the audio file ismapped. The playback file can identify mappings between audio files / note groups and buttons 204. Alternatively or additionally, mappings can be retrieved from storage device 206. In some examples, application 302 can be used to establish mappings between audio files / note groups and instrument buttons or other controls.

[0034] As shown in the example depicted in Fig. 3, user interface 300 can display a strum interface 308 which can be engaged to play back audio files inserted into playline 306. Strum interface 308 provides an area within user interface 300 that can be interacted with via an input device to effect audio file playback. To this end, Fig. 3 shows a cursor 310 controlled by an input device (e.g., mouse, controller joystick, touch input applied to a touch sensor) whose entry and exit into and out of the area of strum interface 308 causes audio file playback. For example, an audio file can be played back upon cursor 310 entering then exiting the area of strum interface 308, at which point the next audio file in the sequence established via playline 306 is queued for playback. Playback can be disabled while cursor 310 is within the area of strum interface 308, with playback being enabled after exit from the area. In this example, the sequence in which inserted audio files are played back is established via playline 306, with the timing of audio file playback being controlled by user input. Such an approach can provide an intuitive mechanism for reviewing a composition, as chosen, or before finalization for upload to an instrument where the composition can be performed.

[0035] While the examples described above regarding application 302 relate to a guitar, audio files can be provided for any suitable type of musical instrument for composition and playback. In these examples, audio files can be recorded with a sound targeting an identified instrument — e.g., as achieved by recording note groups performed on that type of instrument. Where audio files are provided that capture articulations, such articulations can be articulations related to the identified instrument. Further, examples are contemplated in which audio files are provided for multiple, different types of instruments to enable the composition of multiple parts. Different playback files can be provided on a per-instrument / part basis, or playback files can sequence audio files for multiple instrum ents / parts.

[0036] In some examples, application 302 can be used to record the performance of a composition created using the application. Further, examples are contemplated in whichapplication 302 is configured to generate tablature, sheet music, and / or other representations of a composition created with the application. In such examples, user input can be provided to generate representations that include timing-related information, such as tempo, time signature, and / or timings of note-group playback effected by user input or user engagement with an instrument. Such representations can enable other musicians to perform the composition, including players of typical musical instruments and instruments configured with audio file playback functionality as described herein.

[0037] Fig. 4 schematically depicts an example system 400 for recording audio files that capture note groups, generating packet files that comprise collections of audio files, and creating playback files that define a sequence for playback of a collection of audio files. Audio system 100 and / or application 302 can implement aspects of system 400, for example.

[0038] System 400 includes a recording module 402 configured to derive a reduced set of note groups based on a recorded set of note groups captured in audio received at the system. In some examples, this reduction process can be performed to identify the set of note groups in a given part of an audio recording that are representative of the harmonic and rhythmic content of the part and can be used to reproduce the part, while discarding note groups that are duplicative or otherwise exhibit a threshold level of overlap or similarity in harmonic and / or rhythmic content. Such note groups can include repeated note groups in the audio — e.g., groups that repeat intervals, rhythms, and / or articulations.

[0039] As one example reduction, system 400 can reduce two note groups in a recorded set of note groups into a reduced note group where it is determined that a first note group of the two note groups can be generated from a second note group of the two note groups. For example, the two note groups may both capture a bend between two notes, where the magnitude of the bend and duration of the bend in both note groups is within a threshold degree of similarity. However, the intervals at which the bend starts and ends may differ between the two note groups. In this example, pitch-shifting and / or other suitable audio processing applied via system 400 can be used to generate one bend from the other bend. As such, the reduced set can include a note group capturing one bend while omitting a note group capturing the other bend. Another example reduction can include identifying, among the recorded set of notegroups in the analyzed audio, a note group having a relatively longer duration from which a note group having a relatively shorter duration can be generated. In this example, a first note group can be identified that ends with an abrupt stop, from which a second note group can be generated with a shorter duration than that of the first note group.

[0040] For a terminal note (e.g., last note) in a note group of a reduced set, recording module 402 can identify a terminating condition of the terminal note. Identifying the terminating condition can include determining whether the terminal note comes to an abrupt stop, exhibits a smooth transition into a subsequent note group or exhibits a gap before the subsequent note group, and / or includes an articulation (e.g., slide), as examples. Where a terminal note of a note group in the reduced set is identified as exhibiting a first predetermined terminating condition, a first audio file can be recorded with a first temporal property. On the other hand, where a terminal note of a note group in the reduced set is identified as exhibiting a second predetermined terminating condition, a second audio file can be recorded with a second temporal property different from the first temporal property. For example, where the terminating condition is identified as an abrupt stop (e.g., due to the terminal note being played in staccato fashion), a note-group capturing audio file can be recorded with the temporal property of the note group being terminated with an abrupt stop (e g., the note group or terminal note therein being recorded with a duration less than or equal to a threshold duration). In another example, where one note group is terminated with a slide into another note group, an audio file can be recorded with a duration based on the duration of the one note group (e.g., a duration less than or equal to a threshold difference from the duration of the one note group). Conversely, where the terminating condition is identified as a smooth transition, hammer-on, or pull-off into a subsequent note group, or exhibits less than a threshold gap before the subsequent note group, a note-group capturing audio file can be recorded with the temporal property of the note group having a predetermined duration (e.g., between 7 and 12 seconds). At playback time, this approach can enable the terminal note of such an audio group to ring out, which can support a user experience of feeling in control of the timing at which note groups are played back. For example, the terminal note can be allowed to ring out up to the expiration of the threshold duration as long as a button mapped to the note group is engaged,whereas disengagement of the button and / or engagement of another button before expiration of the duration can cease playback of the terminal note. It will be understood that the described approaches can be adapted not just to guitars, but to other instruments and their associated articulations.

[0041] System 400 can implement any suitable mechanisms for deriving reduced sets of note groups and identifying note groups, including intervals, rhythms, terminating conditions, durations, and articulations. For example, system 400 can implement waveform analysis 404, which can include analyzing the amplitude of an audio waveform or note groups therein as a function of time to determine a terminating condition or other temporal properties, and / or pitch detection 406, which can include spectral decomposition or other pitch / frequency analysis to identify intervals and harmonic content. In some examples, one or both of waveform analysis 404 and pitch detection 406 can be used to identify articulations that produce a change from one interval to another.

[0042] Via the modules and techniques described herein, system 400 can identify one or more recorded audio properties 408 of analyzed audio and note groups therein. In some examples, such analysis can be used to identify a target sound (e.g., of a part played by an instrument) exhibited in the analyzed audio. Analyzed audio properties 408 can include but are not limited to frequency properties (e.g., values in bass, mid, and / or treble ranges or other suitable frequency ranges, compression values), gain properties (e.g., gain values exhibited by a distorted guitar part), and effect properties (e.g., delay, reverb, modulation settings and / or values). Identifying analyzed audio properties can also include identifying an instrument and / or aspects relating to the physical configuration of the instrument and / or associated amplification, effects, and / or other devices in a signal chain producing output from the instrument — for example, identifying an instrument / device type, brand, string properties (e.g., gauge, material), and / or pickup type. In a recording process described below, following analysis of recorded audio and reduction of note groups, audio files can be recorded with audio properties based on the audio properties and / or target sound of the analyzed audio.

[0043] Recording module 402 can be used to record a corresponding audio file for each note group in the reduced set of note groups previously identified as described above. To thisend, recording module 402 can implement and / or interface with suitable device(s) and / or module(s), including but not limited to recording equipment (e.g., microphones), mastering and equalizer equipment, effects and other audio processors, and / or amplifiers. Further, in some examples recording module 402 can implement modeling 410 to model a target sound identified in analyzed audio — e.g., amplifier, instrument, and / or effect modeling.

[0044] Audio files can be recorded, for each note group in the reduced set, in any suitable manner. For example, recording can include capturing live performance and / or recording performance by a synthesizing instrument or other programmable device. Each audio file can be recorded with one or more packet audio properties 412 based at least on the recorded audio properties 412 identified from the previously analyzed audio. As described below, audio files recorded via recording module 402 can be bundled in a packet file; as such, audio properties of recorded audio files are referred to in this example as “packet” audio properties 412. Packet audio properties 412 can include but are not limited to amplitude properties (e.g., gain, levels per se), frequency properties (e.g., values in bass, mid, and / or treble ranges or other suitable frequency ranges, compression values, other equalization properties), and effect properties (e.g., the application of and / or values of delay, reverb, modulation, and / or other effects).

[0045] Audio files that capture additional note groups can be generated from the recorded note groups. For example, recording module 402 can implement pitch-shifting 414 to generate additional intervals from intervals captured in the recorded note groups, and / or duration synthesis 416 to generate, from one note group having a first duration, another note group having a second (e.g., shorter) duration. An audio file can be recorded or generated for each note group in such an expanded note group.

[0046] Audio files recorded via recording module 402 can capture any suitable type of note groups having any suitable number of notes, with any suitable harmonic and rhythmic content. As examples, note groups can be recorded that include single notes, double notes, triplets, and other number groupings; whole, half, quarter, eighth, sixteenth, thirty-second and other rhythmic groupings, and / or note groups exhibiting articulations such as hammer-ons, pull-offs, slides, bends, harmonics (e.g., artificial harmonics, pinch harmonics), vibrato, tremolo, and muting / muffling. In some examples, audio files can be recorded for divided note groups. Asparticular examples: a note group comprising eighth notes can be divided into four note groups each having two notes, with an audio file being recorded for each of the four note groups; a note group comprising sixteenth notes can be divided into eight note groups each having two notes, with an audio file being recorded for each of the eight note groups; and a note group comprising thirty-second notes can be divided into sixteen note groups each having two notes, with an audio file being recorded for each of the sixteen note groups. In further examples, note groups (e.g., notes having a common set of intervals and / or rhythms) can be recorded in various durations (e.g., cut, short, medium, long) to provide a set of audio files with a variety of durations, which can enhance the range of selection of audio files for composition and playback, and provide an adaptive quality to the audio file selection process.

[0047] System 400 includes a packet module 418 configured to create a packet file 420 that includes a corresponding audio file 422 for each note group among the previously identified reduced set of note groups. As described above where a part (e.g., solo) in the analyzed audio is targeted in the note group reduction and recording process, audio files 422 provided in the packet file 420 can capture note group(s) that provide the harmonic and rhythmic content for performing the part and its harmonic and rhythmic content through playback of the audio files. In some examples, audio files 422 can include additional harmonic and / or rhythmic content beyond that exhibited by the targeted part to enable such additional content while being potentially in the sound and / or style of the targeted part. In other examples, audio files 422 can comprise harmonic and / or rhythmic content not derived from a targeted part. Audio files 422 can be created and stored in any suitable format, such as a lossless format (e.g., WAV format). Further, in some examples packet file 420 and audio files 422 therein can be encrypted (e.g., via a file module described below). This can enable packet file 420 purchase by end users in a user account-specific manner. Further, where packet file 420 is uploaded to an instrument for playback, audio files 422 can be stored in an encrypted manner — for example, encrypted for that instrument, such that another instrument is unable to decrypt and thus play back the audio files.

[0048] System 400 includes a composition module 424 configured to enable compositions of audio files 422. Via composition module 424, system 400 can receive user input that selectsaudio files and places audio files in a sequence for playback — for example, through user interface 300 presented by application 302. With reference to packet file 420 and application 302, composition module 424 can be used to receive a user input that identifies a selected audio file among audio files 422 in the packet file, and in response insert the selected audio file into playline 306 displayed on interface 300. Further, a user input can be received that causes playback of the selected audio files, and potentially other audio files, inserted into playline 306.

[0049] System 400 includes a file module 426 configured to create a playback file 428 in a file format that identifies a sequential order 430 for playback of selected audio files 422. Sequential order 430 can include the order established by inserting the selected audio files 422 into playline 306 of application 302, for example. However, the file format may not identify a timing (and / or other time-related information such as a tempo or time signature) at which to play back the selected audio files 422. As described above, the timings at which audio files 422 are played back can correspond to user input provided to an application interface, such as cursor input applied to strum interface 308 of application 302, or user engagement of an instrument button or other control such as a virtual button 204 of instrument 200 to which audio file(s) are mapped.

[0050] In some implementations, the file format of playback file 428 can be configured as a text file format. For example, sequential order 430 can comprise an ordered list of entries that each include one or more of an identifier of an associated audio file 422 (e.g., numerical identifier, file name), a button or other instrument control mapped to the associated audio file, and information describing properties of the entry and / or associated audio file such as whether the audio file comprises multichannel audio data. In a particular example where playback file 428 and associated audio files 422 are uploaded to an instrument for playback, a counter that is incremented to iterate through sequential order 430 can be established at the instrument. It will be understood, however, that playback file 428 can be configured with any suitable file format or to identify any suitable type of order for audio file playback. Further, in some examples, audio files 422 selected for upload to an instrument (e.g., to storage device 206 of instrument 200) can be stored as individual audio files, for example, and not as a collectivesong, part, performance, or recording. Further, while performances through audio file 422 playback at such an instrument are contemplated, such performances in some examples may not be recorded, stored, and / or reproduced at / from the instrument.

[0051] The approaches described herein can enable music composition and performance with musical building blocks in the form of audio files that provide harmonic and rhythmic variety, without burdening users with the demands of proficiency on a typical musical instrument or knowledge of music theory or harmony. Instead, composition and playback can be enabled through the visual and / or acoustic selection and sequencing of audio files. As described above, the disclosed approaches can enable freeform composition, or composition and / or performance targeting a part in recorded audio. Where a part is targeted, audio files can be provided that include harmonic and rhythmic content to perform the targeted part. Additionally, examples are contemplated in which additional harmonic and / or rhythmic content is provided to enable composition in the style and / or sound of the targeted part. Yet other examples are contemplated in which multiple instruments configured with audio file playback capability described herein can be used to achieve a multi -instrument performance in which different instruments perform different parts (e.g., rhythm and lead parts).

[0052] Fig. 5 depicts a flowchart illustrating an example method 500 of recording audio files for note groups in a reduced set of note groups. Method 500 can be implemented at system 100 and / or system 400, as examples.

[0053] At 502, method 500 includes deriving a reduced set of one or more note groups based at least in part on a recorded set of note groups each having one or more notes. Each recorded set of note groups can have one or more recorded audio properties. At 504, method 500 includes identifying a terminating condition of a terminal note in the reduced set. At 506, method 500 includes recording a first audio file responsive to identifying that the terminal note exhibits a first predetermined terminating condition, the first audio file being recorded with a first temporal property and capturing a first note group of the reduced set. At 508, method 500 includes recording a second audio file responsive to identifying that the terminal note exhibits a second predetermined terminating condition, the second audio file being recorded with a second temporal property different from the first temporal property and capturing a secondnote group of the reduced set. At 510, method 500 includes creating a packet file that includes a corresponding audio file for each note group in the reduced set, the packet file including at least the first and second audio files, each corresponding audio file being recorded with one or more packet audio properties selected based at least on the one or more recorded audio properties. In some examples, method 500 can end following 510. In other examples, method 500 can optionally continue to 602 of a method 600 illustrated in Fig. 6.

[0054] Fig. 6 depicts a flowchart illustrating an example method 600 of creating a file for playback of audio files. Method 600 can be implemented at system 100, application 302, and / or system 400, as examples.

[0055] At 602, method 600 includes receiving a first user input that identifies a selected one or more of the audio files in a packet file. At 604, method 600 includes inserting the selected one or more audio files into a playline displayed on an interface. At 606, method 600 includes receiving a second user input to cause playback of the selected one or more audio files inserted into the playline. At 608, method 600 includes receiving a third user input to cause creation of a file in a file format that identifies a sequential order for playback of the selected one or more audio files inserted into the playline and does not identify a timing at which to play back the selected one or more audio files.

[0056] In some examples, the methods, modules, and processes described herein can be tied to a computing system of one or more computing devices. Such methods and processes can be implemented as a computer application program or service, an applicationprogramming interface (API), a library, and / or other computer-program product.

[0057] FIG. 7 schematically shows an example computing system 700 that can enact one or more of the methods and processes described above. Computing system 700 can take the form of one or more personal computers, server computers, tablet computers, homeentertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and / or other computing devices.

[0058] Computing system 700 includes a logic subsystem 702 and a storage subsystem 704. Computing system 700 can optionally include an input subsystem 706, output subsystem 708, and / or other components not shown in FIG. 7.

[0059] Logic subsystem 702 includes one or more physical devices configured to execute instructions. For example, the logic subsystem can be configured to execute instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions can be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.

[0060] The logic subsystem can include one or more processors configured to execute software instructions. Additionally or alternatively, the logic subsystem can include one or more hardware or firmware logic subsystems configured to execute hardware or firmware instructions. Processors of the logic subsystem can be single-core or multi-core, and the instructions executed thereon can be configured for sequential, parallel, and / or distributed processing. Individual components of the logic subsystem optionally can be distributed among two or more separate devices, which can be remotely located and / or configured for coordinated processing. Aspects of the logic subsystem can be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.

[0061] Storage subsystem 704 includes one or more physical devices configured to hold instructions executable by the logic subsystem to implement the methods and processes described herein. In some examples, implementation of methods and processes can transform the state of storage subsystem 704 — e.g., to hold different data.

[0062] Storage subsystem 704 can include removable and / or built-in devices. Storage subsystem 704 can include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and / or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage subsystem 704 can include volatile, nonvolatile, dynamic, static, read / write, read-only, random-access, sequential-access, location-addressable, file-addressable, and / or content-addressable devices.

[0063] It will be appreciated that storage subsystem 704 includes one or more physical devices. However, aspects of the instructions described herein alternatively can be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration.

[0064] Aspects of logic subsystem 702 and storage subsystem 704 can be integrated together into one or more hardware-logic components. Such hardware-logic components can include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC / ASICs), program- and application-specific standard products (PSSP / ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.

[0065] The terms “module” and “application” can be used to describe an aspect of computing system 700 implemented to perform a particular function. In some cases, a module or application can be instantiated via logic subsystem 702 executing instructions held by storage subsystem 704. It will be understood that different modules and / or applications can be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module and / or application can be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms “module” and “application” can encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.

[0066] When included, input subsystem 706 can comprise or interface with one or more input devices such as a keyboard, mouse, touchscreen, or game controller. When included, output subsystem 708 can be used to present a visual representation of data held by storage subsystem 704. This visual representation can take the form of a graphical user interface (GUI) or other interface. Output subsystem 708 can include one or more display devices utilizing any suitable type of technology. Such display devices can be combined with logic subsystem 702 and / or storage subsystem 704 in a shared enclosure, or such display devices can be peripheral display devices. In some examples, output subsystem 708 can be configured to communicatively couple computing system 700 with one or more other computing devices. A communication subsystem can include wired and / or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem can be configured for wired or wireless communication. In some examples, the communication subsystem can allow computing system 700 to send and / or receive messages to and / or from other devices via a network such as the Internet.

[0067] It will be understood that the configurations and / or approaches described herein are exemplary in nature, and that these specific examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein can represent one or more of any number of processing strategies. As such, various acts illustrated and / or described can be performed in the sequence illustrated and / or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes can be changed.

[0068] The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and / or properties disclosed herein, as well as any and all equivalents thereof.

Claims

CLAIMSWhat is claimed is:

1. A method, comprising: deriving a reduced set of one or more note groups based at least in part on a recorded set of note groups each having one or more notes, each recorded set of note groups having one or more recorded audio properties; identifying a terminating condition of a terminal note in the reduced set; recording a first audio file responsive to identifying that the terminal note exhibits a first predetermined terminating condition, the first audio file being recorded with a first temporal property and capturing a first note group of the reduced set; recording a second audio file responsive to identifying that the terminal note exhibits a second predetermined terminating condition, the second audio file being recorded with a second temporal property different from the first temporal property and capturing a second note group of the reduced set; creating a packet file that includes a corresponding audio file for each note group in the reduced set, the packet file including at least the first and second audio files, each corresponding audio file being recorded with one or more packet audio properties selected based at least on the one or more recorded audio properties; receiving a first user input that identifies a selected one or more of the audio files in the packet file; inserting the selected one or more audio files into a playline displayed on an interface; receiving a second user input to cause playback of the selected one or more audio files inserted into the playline; and receiving a third user input to cause creation of a file in a file format that identifies a sequential order for playback of the selected one or more audio files inserted into the playline and does not identify a timing at which to play back the selected one or more audio files.

2. The method of claim 1, wherein the first note group of the reduced set and the second note group of the reduced set each include one or more of an interval, a rhythm, and an articulation.

3. The method of claim 1 , wherein the first predetermined terminating condition includes an abrupt stop, and the first temporal property includes an abrupt stop.

4. The method of claim 1, wherein the second predetermined terminating condition includes a smooth transition into a subsequent note group or an articulation, and the second temporal property includes a predetermined duration.

5. The method of claim 1, wherein the one or more recorded audio properties and the one or more packet audio properties include one or more of a frequency property, a gain property, and an effect property.

6. The method of claim 1, further comprising encrypting the selected one or more audio files.

7. The method of claim 1, wherein each audio file of the selected one or more audio files is configured for playback at a timing determined by user input.

8. The method of claim 7, wherein the user input is received via an input device at the interface.

9. The method of claim 7, wherein the user input is received at a control of an instrument.

10. The method of claim 1, further comprising obtaining an expanded set of note groups from the reduced set of one or more note groups via one or both of pitch-shifting and duration synthesis, and recording an audio file for each note group in the expanded set of note groups.

11. A system, comprising: an input subsystem configured to receive audio including a recorded set of note groups each having one or more notes, each recorded set of note groups having one or more recorded audio properties; an output subsystem configured to display an interface; a logic subsystem; and a storage subsystem comprising one or more modules executable by the logic subsystem, the one or more modules including: a recording module configured to derive a reduced set of one or more note groups based at least in part on the recorded set of note groups, identify a terminating condition of a terminal note in the reduced set, record a first audio file responsive to identifying that the terminal note exhibits a first predetermined terminating condition, the first audio file being recorded with a first temporal property and capturing a first note group of the reduced set, and recording a second audio file responsive to identifying that the terminal note exhibits a second predetermined terminating condition, the second audio file being recorded with a second temporal property different from the first temporal property and capturing a second note group of the reduced set; a packet module configured to create a packet file that includes a corresponding audio file for each note group in the reduced set, the packet file including at least the first and second audio files, each corresponding audio file being recorded with one or more packet audio properties selected based at least on the one or more recorded audio properties; a composition module configured to receive a first user input that identifies a selected one or more of the audio files in the packet file,insert the selected one or more audio files into a playline displayed on the interface, and receive a second user input to cause playback of the selected one or more audio files inserted into the playline; and a file module configured to create a file in a file format that identifies a sequential order for playback of the selected one or more audio files inserted into the playline and does not identify a timing at which to play back the selected one or more audio files, and store the file at the storage subsystem.

12. The system of claim 11, wherein the first note group of the reduced set and the second note group of the reduced set each include one or more of an interval, a rhythm, and an articulation.

13. The system of claim 11, wherein the first predetermined terminating condition includes an abrupt stop, and the first temporal property includes an abrupt stop.

14. The system of claim 11, wherein the second predetermined terminating condition includes a smooth transition into a subsequent note group or an articulation, and the second temporal property includes a predetermined duration.

15. The system of claim 11, wherein the one or more recorded audio properties and the one or more packet audio properties include one or more of a frequency property, a gain property, and an effect property.

16. The system of claim 11, wherein the file module is further configured to encrypt the selected one or more audio files.

17. The system of claim 11, wherein each audio file of the selected one or more audio files is configured for playback at a timing determined by user input.

18. The system of claim 17, wherein the user input is received via an input device at the interface.

19. The system of claim 17, wherein the user input is received at a control of an instrument.

20. The system of claim 11, wherein the recording module is further configured to obtain an expanded set of note groups from the reduced set of one or more note groups via one or both of pitch-shifting and duration synthesis, and to record an audio file for each note group in the expanded set of note groups.

Images