Track separation system and device
By using a compressed track splitting module and feature learning technology on user electronic devices, the computational requirements for audio track splitting in resource-constrained environments are addressed, achieving efficient audio track splitting and playback.
Patent Information
- Application Number
- CN202580015979.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2025-02-24
- Filing Date
- 2025-02-25
- Publication Date
- 2026-11-17
AI Technical Summary
Existing track separation algorithms consume a lot of computing resources in resource-constrained environments, making it difficult to efficiently perform audio track separation and playback on user electronic devices.
It employs a compressed, compact, and lightweight track separation module, combined with a neural processing unit and a central processing unit, to separate and play back audio signals in hardware-constrained environments. It also utilizes feature learning and temporal information processing techniques to reduce computational requirements.
It enables efficient audio track splitting and playback in resource-constrained environments, reducing computation and power consumption, and is suitable for various audio processing scenarios.
Smart Images

Figure CN123125572A_ABST
Abstract
Description
Cross-references to related applications
[0001] This application claims U.S. Provisional Patent Application No. 63 / 708,164, filed October 16, 2024, entitled "STEM SEPARATION SYSTEMS AND DEVICES"; U.S. Utility Model Patent Application No. 19 / 061,111, filed February 24, 2025, entitled "STEM SEPARATION SYSTEMS AND DEVICES"; and U.S. Provisional Patent Application No. 63 / 558,985, filed February 28, 2024, entitled "STEM SEPARATION SYSTEMS AND DEVICES". The priority of U.S. Utility Model Patent Application No. 19 / 061,152, entitled “ANDDEVICES”, the entirety of which is incorporated herein by reference for all purposes. Background Technology
[0002] Audio processing involves manipulating, optimizing, transforming, and / or extracting information from audio signals. In the music industry, audio processing plays a crucial role in shaping and improving music quality. Audio processing is also conducted in a variety of other fields, such as film and television, broadcasting and radio, telecommunications, speech recognition and synthesis, gaming, and / or other areas.
[0003] The subject matter claimed herein is not limited to embodiments that operate only in environments such as those described above. Rather, this background is provided merely to illustrate an exemplary technical field in which some of the embodiments described herein can be practiced. Attached Figure Description
[0004] To describe how the above and other advantages and features can be obtained, a more specific description of the subject matter briefly described above will be presented by referring to specific embodiments shown in the accompanying drawings. It should be understood that these drawings depict only exemplary embodiments and are therefore not to be considered as limiting the scope; the embodiments will be described and explained in more specific and detailed manner by means of the drawings, in which: Figure 1A , 1B Figures 1C and 1D illustrate conceptual representations of example components, elements, and actions associated with device-driven stem separation in a playback environment.
[0005] Figure 2 and 3An example flowchart depicting actions associated with the disclosed topic is shown.
[0006] Figures 4 to 11 This demonstrates a sample user interface front-end that can facilitate or convey information related to system-level track separation functionality.
[0007] Figure 12 Example components that may contain or be configured to perform various embodiments of a system are described. Detailed Implementation
[0008] The disclosed embodiments relate to systems and apparatus for facilitating track separation.
[0009] As mentioned above, audio processing is performed in various fields and involves manipulating, optimizing, transforming, and / or extracting information from audio signals. Audio track splitting (or simply "track splitting") is a type of audio processing that involves separating audio into its basic components or "tracks," which correspond to various types of sounds represented in the audio, such as vocals, drums, bass, strings, piano / keyboard instruments, melodies, dialogue, effects, background music, unclassified sounds, etc. Track splitting is performed in various fields, such as music production, music education, creating karaoke tracks, and forensic audio analysis.
[0010] Conventional track separation algorithms can analyze and separate individual audio tracks from a single audio file, relying on pattern recognition and spectral analysis to isolate sound sources from the audio file based on unique characteristics such as frequency and amplitude. Many conventional track separation algorithms utilize artificial intelligence (AI) techniques (e.g., using deep learning and neural networks) to improve the isolation of different sound sources from audio tracks, where different sound sources have overlapping frequencies (which can occur simultaneously in the audio tracks).
[0011] Conventional track splitting models are typically offered as cloud services, where users can submit jobs specifying one or more audio tracks to be processed using a cloud-intensive track splitting model. The split audio output (including individual audio tracks from the input audio tracks) is then provided to the requester (e.g., as a downloadable file).
[0012] Conventional track separation models typically consume significant amounts of power and computing resources, and are therefore implemented in resource-intensive environments (e.g., using graphics processing units (GPUs) residing on cloud servers).
[0013] At least some of the disclosed embodiments relate to means configurable to perform audio track separation on an input audio signal while outputting a track-separated audio signal for playback by one or more playback components. Implementations of the embodiments disclosed herein enable the isolation and playback of audio tracks from an input audio signal (not limited to a complete audio file) in resource-constrained environments, such as on a user's electronic device. The means for facilitating track separation in a playback environment, as described herein, may include one or more processing units and one or more computer-readable recording media (e.g., computer memory). The processing unit may include one or more central processing units, neural processing units, graphics processing units, and / or other types of processing circuitry systems.
[0014] The device can receive an input audio signal (e.g., a digital audio signal from any source, such as a file, stream, radio, line input, analog-to-digital converter, or other source) and identify a first set of segments from the input audio signal. The first set of segments may include one or more audio segments of the input audio signal having one or more specified durations (e.g., where the segments have individual or aggregated durations ranging from about half a second to about four seconds, or in some instances, durations greater than four seconds or less than half a second). For example, the device may receive the input audio signal over time (e.g., in the case of a line input connection or radio, streaming, television broadcast, or other media playback signal transmission mode), and may define the first set of segments as the device receives the input audio signal (e.g., defining each temporal second (or other duration) of the received audio signal as a separate set of one or more audio segments).
[0015] After the first set of audio segments is defined, the device can use a track splitting module to process the first set of audio segments, which can provide the first set of track-split segments (the set includes one or more track-split segments). The first set of track-split segments can include multiple audio tracks corresponding to different audio sources represented in the first set of audio segments (e.g., vocals, bass, drums, guitar, strings, piano / keyboard instruments, wind instruments, noise, sound effects, other / residual audio).
[0016] Track splitting modules may be stored locally on the device and may include compressed, compact, lightweight, embedded, or mobile track splitting modules suitable for implementation in hardware / resource-constrained environments, as described in more detail below. In some implementations, the track splitting modules are selected from a set of track splitting modules or a library of track splitting modules stored on the device. Track splitting modules may be selected based on one or more configurations, preferences, settings, or contexts of the current track splitting session. For example, in conjunction with activating a track splitting mode for the device, a user may indicate one or more of the following via user input: (i) which audio tracks are present in the input audio signal (e.g., vocals, bass, drums, guitar, strings, piano / keyboard instruments, wind instruments, and / or other tracks); or (ii) which / which audio tracks from the input audio signal are to be isolated for playback. The device may utilize such indications to select the track splitting module for a particular track splitting session. For example, the device can store multiple track splitting modules suitable for use with certain media types (e.g., music or different genres of music, audiovisual content such as movies or video game content with accompanying audio), for use with audio signals containing certain audio tracks, or for outputting certain audio tracks (or combinations of audio tracks) for playback. The device can utilize user instructions provided through the aforementioned user input (e.g., via a lookup table or other search / selection method) to select the track splitting module to be used in the current (or future) track splitting session.
[0017] Alternatively or concurrently, the device may perform preprocessing on an initial segment of the input audio signal to determine identification information of the audio signal or to determine which audio tracks are present in the audio signal. Such information obtained by preprocessing the initial segment of the input audio signal can be used to enable the device to automatically select a track separation module (e.g., via a lookup table or other search / selection method) for the current (or future) track separation session.
[0018] After processing a first set of audio segments of the input audio signal through a (selected) track separation module to obtain a first set of track-separated segments, the device can induce playback of at least one selected audio track from the first set of track-separated segments. For example, user input, preferences, or settings can specify desired audio tracks for playback (e.g., vocals, bass, drums, guitar, strings, piano / keyboard instruments, wind instruments, other / remaining audio), and the device can induce playback of the selected audio tracks from the first set of track-separated segments corresponding to the desired audio tracks. The device can induce playback of the selected audio tracks by converting them into analog signals and providing the analog signals to speakers (e.g., to speakers on the device or to individual or external speakers via a line-out connection). Alternatively or additionally, the device can send a digital representation of the selected audio tracks from the first set of track-separated segments to a playback device (e.g., via a digital interface or wireless connection) to facilitate playback of the selected audio tracks by the playback device (wherein the analog conversion may occur at the playback device). Playback of selected audio tracks may include selectively avoiding playback of unselected audio tracks (e.g., the device may cause playback of vocal tracks only when other audio tracks are avoided).
[0019] During the playback of a selected audio track in the first set of track-separated segments (or during the processing of the first set of segments from the input audio signal by the track separation module to obtain the first set of track-separated segments), the device may identify a second set of segments (one or more second segments) from the input audio signal, such as by continuing to receive the input audio signal over time and defining the second set of segments when the device receives the input audio signal (e.g., defining the second set of segments as a time second (or other duration) after a first time second (or other duration) of the received audio signal).
[0020] When a second set of audio segments is defined, the device can process the second set of audio segments to obtain a second set of track-separated segments (one or more second track-separated segments), which may include multiple audio tracks corresponding to different audio sources represented in the second set of audio segments. The second set of audio segments may be processed serially by the track separation module and the processing of the first set of audio segments (e.g., after the processing of the first set of audio segments by the track separation module is completed, or during the playback of the first set of audio segments), or at least partially in parallel with the processing of the first set of audio segments (e.g., in the case where the processing of the second set of audio segments is initiated before the processing of the first set of audio segments by the track separation module is completed, which may depend on the hardware capabilities of the device).
[0021] After the playback of the selected audio tracks of the first set of separated audio segments is completed, the device can trigger the playback of the selected audio tracks of the second set of separated audio segments (which may correspond to the same audio source as the selected audio tracks of the first set of separated audio segments). The playback of the selected audio tracks of the second set of separated audio segments can be implemented in a manner similar to that described above for the playback of the selected audio tracks of the first set of separated audio segments.
[0022] In some instances, the device may play back the input audio signal without performing track separation before initiating the track separation mode (e.g., by passing the input audio signal to one or more devices or external playback components, which may include intermediate processing / transformation such as analog-to-digital conversion or digital-to-analog conversion, encoding / decoding, compression / decompression, wireless or wired data transmission, etc.). During the playback of the input audio signal, the device may receive user input for activating the track separation mode. User input may take any suitable form (e.g., through user interaction with user interface hardware such as a touchscreen, mouse, keyboard or other controller, button / switch / knob, microphone for voice input, image sensor for gesture input, etc.). After detecting user input for activating the track separation mode, the device may avoid continuing the playback of the input audio signal and may activate the track separation mode to begin track separation processing to facilitate the playback of one or more individual audio tracks of the input audio signal. In some instances, the track splitting module to be used for track splitting processing is determined based on user input and / or based on the preprocessing of the input audio signal (e.g., before or after activating the track splitting mode).
[0023] When the device operates in track splitting mode, the following actions can be performed iteratively until a stopping condition is met: (i) identifying an audio segment (or a set of audio segments) from the input audio signal; (ii) using the track splitting module to process the audio segment (or a set of audio segments) to obtain a track-split segment (or a set of track-split audio segments); and (iii) causing playback of one or more audio tracks of the track-split segment. Within each iteration, actions (i) and / or (ii) above can be performed on the current audio segment during processing of a previously identified audio segment or during playback of the audio tracks of a previously generated track-split segment. Within each iteration, action (iii) above can be performed after (or during) the completion of playback of the audio tracks of a previously generated track-split segment. Action (iii) above can include avoiding playback of one or more remaining audio tracks (e.g., to isolate or remove vocals and / or one or more types of user instruments). In some embodiments, one or more audio splicing operations are performed to combine consecutively generated audio tracks for playback. In some instances, track splitting is performed to facilitate the playback of one or more individual audio tracks by associating or synchronizing the input audio signal with the input video signal. The playback of the video signal can be delayed to be time-synchronized with the playback of the individual audio tracks facilitated by the track splitting mode.
[0024] After the stopping conditions are met, the system can deactivate the track splitting mode, and in some instances, it can revert to playback of the input audio signal without applying track splitting to it.
[0025] Stop conditions used to trigger the deactivation of the track splitting mode can take various forms. For example, a stop condition may include detecting user input (any type of user input) for deactivating the track splitting mode. In some implementations, a stop condition may include performing a predetermined number of track splitting iterations, or may include satisfying other metrics, values, or thresholds (e.g., the number of times or duration the track splitting module runs, the number of audio segments identified or processed from the input audio signal, the number or duration of split audio tracks being played back or queued for playback, the number or duration of audio tracks or media content items for which track splitting is performed, the amount of time the input audio signal is processed, the amount of time the split audio signal is played back, the amount of time spent in the track splitting mode, and / or others).
[0026] Stop conditions or associated metric / value thresholds for deactivating a track separation mode can be defined, at least in part, based on the service level associated with the track separation software / model stored on the device. For example, the device may comprise a consumer electronics device (e.g., a speaker, musical instrument, television, home theater system, car audio system, amplifier or smart amplifier, mobile electronics device, etc.), and a limited, trial, or restricted version of the track separation software / model may initially be ported to the electronic device (e.g., at the manufacturer / developer level). For example, the manufacturer or developer may access a limited version of the track separation software / model through a third-party software library for implementation using an electronic device manufactured by the manufacturer or developer (e.g., after verifying that the electronic device provides sufficient hardware support for the operation of the track separation software / model). The limited version of the track separation software / model can restrict the operation of the track separation model by imposing one or more stop conditions among the stop conditions for the operation of the track separation mode (e.g., as described above, causing the deactivation of the track separation mode after one or more thresholds are met). A full version of the track splitting software / model can then be ported to the electronic device (e.g., after a trial period, after licensing, after the end user and / or manufacturer subscribes to or compensates a third party, etc.). The full version of the track splitting software / model can omit the limitations associated with the limited version. For example, stop conditions based on metrics, values, or thresholds associated with the limited version can be omitted from the full version (where the primary stop condition is the deactivation of the user-guided track splitting mode). In some instances, the full version of the track splitting software / model is initially ported to the electronic device at the manufacturer / developer level but remains in a restricted, locked, or limited state until further action is taken by the end user or manufacturer.
[0027] Having already described some of the advanced features and benefits among the various advanced features and benefits of the disclosed embodiments, attention now turns to the accompanying drawings, which illustrate various conceptual representations, architectures, methods, and / or supporting illustrations in relation to the disclosed embodiments.
[0028] Figure 1A , 1B Figures 1C and 1D illustrate conceptual representations of example components, elements, and actions associated with device-driven track separation in a playback environment. Specifically, Figure 1A Example aspects of a device 100 for facilitating track separation in a playback environment are shown. Device 100 may include consumer electronic devices, such as speakers, televisions, home theater systems, car audio systems, amplifiers or smart amplifiers, mobile electronic devices, and / or others, as non-limiting examples. Device 100 may include, correspond to, or incorporate one or more components of system 1200, as referenced below. Figure 12 As described. Figure 1AThe device 100 includes an input interface 102, which enables the device 100 to receive or otherwise access an input audio signal 104 (in... Figure 1A (represented as a waveform in the text). Figure 1A In some instances, the input audio signal 104 may comprise audio signals of music tracks continuously provided to or accessed / utilized by the device 100 over time, such as via analog or digital broadcasting or streaming (e.g., analog radio, digital radio, satellite radio, the Internet, or network streaming), analog or digital interfaces, and / or other modalities. In some instances, the input audio signal 104 comprises analog signals from analog audio transmissions (e.g., analog audio broadcasting or line-in connections) that are converted into digital signals by the device 100. The input interface 102 for facilitating access to or retrieval of the input audio signal 104 may take any suitable form, such as wireless communication hardware (e.g., Bluetooth, Wi-Fi, or other wireless communication protocols), wired communication hardware (e.g., line-in connections such as 3.5 mm jack (AUX), RCA cables, XLR cables, TRS / TRRS cables, USB audio, HDMI, optical, coaxial), audio channels or buses, and / or others.
[0029] Figure 1A Device 100 is shown as including an output interface 106, whereby device 100 can induce playback of an output audio signal 108. For example, device 100 may receive an input audio signal 104 via input interface 102 and pass the input audio signal 104 to output interface 106 (as indicated by the arrow extending from input interface 102 to output interface 106) to facilitate audio playback (thus generating the output audio signal 108). In some instances, device 100 may perform one or more intermediate operations on the input audio signal 104 received at input interface 102 to generate the output audio signal 108 (e.g., analog-to-digital conversion or digital-to-analog conversion, encoding / decoding, compression / decompression, wireless or wired data transmission, etc.). Output interface 106 may include a speaker of device 100 or a communication channel (analog or digital, and wired or wireless) between device 100 and a separate playback device.
[0030] Figure 1AThe device 100 is also shown to include a track splitting module 110 that can be locally stored on the device 100. The track splitting module 110 may include one or more compressed, compact, lightweight, embedded, or mobile track splitting modules suitable for implementation in hardware / resource-constrained environments. For example, the track splitting module 110 may be configured to analyze short (potentially overlapping) audio frames or segments from the input audio signal 104 (thereby reducing the amount of data processed at any given moment to manageable chunks). The analyzed frames or segments may be temporally adjacent, enabling the formation of temporally continuous outputs of individual audio tracks.
[0031] The segments analyzed by the track separation module 110 can be approximately 1 second long or in the range of approximately 0.5 seconds to approximately 4 seconds or longer (e.g., less than 8 seconds). For each frame or segment of audio identified from the input audio signal 104, the track separation module 110 can extract features associated with the audio track to be separated, such as temporal features / relationships, spectral features, phase information, amplitude information, learned features (e.g., determined by a feature learning model), spatial audio information, and / or other sonic characteristics. In some instances, the track separation module 110 can utilize feature sizes reduced relative to conventional track separation modules, such as 256-channel features, 128-channel features, 64-channel features, etc. The track separation module 110 can use the extracted features to predict the components of each audio track represented in each frame / segment. The track separation module 110 can apply masking or filtering to the frame / segment (or its spectral representation) to isolate each audio track of the frame / segment. The continuous audio of each audio track in the represented audio track can be formed by splicing or reconstructing temporally adjacent audio segments separated by the track (e.g., considering potential temporal overlap between segments).
[0032] In some implementations, the track splitting module 110 is adapted to be used with certain processing units or hardware accelerators (such as neural processing units (NPUs) and central processing units (CPUs)) that may have lower power and / or resource consumption levels than graphics processing units (GPUs). For example, to facilitate processing via one or more NPUs, the track splitting module 110 may be configured to avoid using complex numbers, such as by avoiding conventional techniques used to generate spectrograms of audio frames / segments identified from the input audio signal 104 (e.g., alternatively relying on time-domain information / techniques such as direct model-based predictions in the time domain, using custom time-domain features, mapping time-domain information to a latent space for track splitting, time-domain filtering techniques, etc.). As another example, the track splitting module 110 may be subjected to quantization, where lower bit-width numbers (e.g., 8-bit or 16-bit integers instead of floating-point numbers) are used to represent model parameters, which can reduce model size and / or increase model speed. The model size of the track splitting module 110 can be further reduced by decreasing the number of model layers (e.g., four Transformer layers), which allows the track splitting module 110 to operate on memory-constrained devices (compared to cloud servers that typically use conventional track splitting models). In some implementations, the track splitting module 110 is generated using techniques such as knowledge distillation, weight pruning, neuron pruning, quantization, parameter sharing, factorization, and / or other techniques.
[0033] exist Figure 1A In this example, device 100 operates in the "off" state of the track separation mode (indicated by box 112), where device 100 receives an input audio signal 104 through input interface 102 and provides an output audio signal 108 at output interface 106 (e.g., without utilizing track separation module 110). When operating in the off state of track separation mode, device 100 can determine whether a start condition has been met (indicated by box 114). The start condition may include one or more conditions for activating the track separation mode, such as receiving a user command or detecting the presence of one or more events / states (e.g., characteristics of the input audio signal 104). When the start condition is not met (indicated by the "No" arrow extending from box 114), device 100 can continue operating in the "off" state of track separation mode (indicated by box 112), which may include continuing audio playback without performing track separation. When the start conditions are met (indicated by the "yes" arrow extending from box 114), the device 100 can begin operation with the track separation mode in the "on" state (indicated by box 116).
[0034] Figure 1B An example of device 100 operating in the "On" state of track separation mode is illustrated (indicated by box 118). When operating in the "On" state of track separation mode, device 100 can determine whether a stop condition has been met (indicated by box 120). The stop condition may include one or more conditions for disabling track separation mode, such as receiving a user command, detecting that a allocated or permitted amount of track separation has been performed (or operation when track separation mode is on), and / or other conditions as described above. When the stop condition is not met (indicated by the "No" arrow extending from box 120), device 100 may continue operating in the "On" state of track separation mode (indicated by box 118). When the stop condition is met (indicated by the "Yes" arrow extending from box 120), device 100 may begin operating in the "Off" state of track separation mode (indicated by box 122).
[0035] Based on the operation of device 100 in the "on" state of track separation mode, device 100 can identify segment 124A from input audio signal 104. For example, when device 100 receives input audio signal 104 through input interface 102 over time, device 100 can define a time segment (e.g., a one-second segment or another duration) of the received input audio signal 104 as segment 124A. For illustrative purposes, Figure 1B Conceptually, segment 124A is depicted as a portion of the input audio signal 104 that has traveled through input interface 102 (from left to right). Although Figure 1B Segment 124A is shown as comprising a single time segment, but segment 124A may include multiple constituent segments that together form segment 124A (e.g., constituent segments may overlap in time).
[0036] When the track separation mode is enabled, device 100 can use track separation module 110 to process segment 124A. As described above, track separation module 110 may include multiple track separation modules, and device 100 can select a specific track separation module for use in the current track separation session (e.g., to process segment 124A and / or subsequent segments). Track separation module 110 may include different track separation modules customized for different use cases (e.g., different genres of music, different tracks to be separated / output, different audio sources present in the input audio signal 104, etc.). For example, the track separation module 110 may include: one or more track separation modules configured to identify / separate one or more or a combination of vocal tracks, bass tracks, drum tracks, guitar tracks, string tracks, piano tracks, keyboard instrument tracks, wind instrument tracks, and / or other / remaining audio tracks (e.g., a music track separation module); one or more track separation modules configured to identify / separate one or more or a combination of dialogue tracks, music tracks, and / or effects tracks (e.g., a movie track separation module); one or more track separation modules configured to identify / separate one or more or a combination of lead vocal tracks, harmony tracks, and / or other vocal tracks (e.g., a music track separation module); For example, a vocal track separation module; one or more track separation modules configured to identify / separate one or more of rhythm guitar tracks, solo guitar tracks, and / or other guitar tracks, or combinations thereof (e.g., a guitar vocal track separation module); one or more track separation modules configured to identify / separate one or more of bass drum tracks, snare drum tracks, tom-toms tracks, hi-hat tracks, cymbal tracks, and / or other drum tracks, or combinations thereof (e.g., a drum track separation module); and / or one or more track separation modules configured to identify / separate one or more of acoustic guitar tracks, electric guitar tracks, and / or other guitar tracks, or combinations thereof (e.g., a guitar track separation module).
[0037] Device 100 may select a specific track separation module from the track separation modules 110 for the current track separation session based on user input, such as user input selecting a track separation module from a list of available track separation modules 110, or user input indicating identification information of the audio signal to which track separation will be performed, which audio tracks are present in the input audio signal, which audio tracks to isolate from the input audio signal for playback, and / or other information. In some embodiments, device 100 may select a specific track separation module from the track separation modules 110 for the current track separation session based on preprocessing of an initial segment of the input audio signal 104. Such preprocessing may utilize, for example, a classification module (e.g., SVM, neural network, random forest, and / or others) trained to classify segments of the input audio (and / or features extracted therefrom) to provide one or more labels indicating audio sources present in the input audio. These labels may be used to select a specific track separation module for the current track separation session.
[0038] Figure 1B A conceptual description depicts a track splitting module 110 receiving and processing segment 124A to generate a track-split segment 136A (which may contain multiple component-type track-split segments, such as segment 124A comprising multiple component-type segments). The track-split segment 136A may include audio tracks associated with different audio sources represented in segment 124A processed by the track splitting module 110. For example, Figure 1B The segment 136A, which has been separated into tracks, is displayed as including vocal track 126A, drum track 128A, bass track 130A, guitar track 132A, and other tracks 134A (representing the remaining audio that is not part of other tracks). Figure 1B Each audio track of the separated segment 136A is displayed as a waveform representing the separated audio content adjacent to each other. In view of this disclosure, it should be understood that... Figure 1B The track separation of segment 136A is provided by way of example only and does not limit the disclosed principles (e.g., segment 136A may include additional, fewer or alternative audio tracks).
[0039] Figure 1B An arrow extending from vocal track 126A toward the output interface 106 of device 100 is shown, indicating that device 100 queues vocal track 126A for playback. Vocal track 126A can be selected for playback based on user-defined settings / selections (e.g., the user selects a track separation mode where only vocals from input audio signal 104 are played back). Although Figure 1BOnly a single audio track from the track-separated segment 136A is described as being queued for playback, but according to this disclosure, multiple and / or other audio tracks from the track-separated segment 136A can be played back (or omitted from playback).
[0040] Figure 1C The concept of the vocal track 126A is illustrated by showing it as having passed through output interface 106 (traveling from left to right). Figure 1B Playback of vocal track 126A. Device 100 can induce playback of vocal track 126A in various ways, such as by providing or transmitting the separated audio signal (digital or analog) based on vocal track 126A to one or more speakers of device 100 or to one or more external playback components / devices. Device 100 can selectively avoid inducing the remaining audio tracks that cause the separated segment 136A (e.g., in...). Figure 1B In the example, playback of the drum track 128A, bass track 130A, guitar track 132A, and other tracks 134A.
[0041] Figure 1C The device 100 receiving the input audio signal 104 at the input interface 102 is also conceptually depicted as having progressed in time (e.g., the input audio signal 104 has moved further to the right, with a larger portion of the input audio signal 104 having crossed the input interface 102 from left to right). Figure 1C Depicting the above reference Figure 1B The segment 124A processed by the described track separation module 110 causes, for example, Figure 1C The playback of the vocal track 126A shown is shown. Figure 1C Another segment 124B, identified from the input audio signal 104, is also depicted, which can be identified by the device 100 in a manner similar to that described above for the identification of segment 124A. Segment 124B can be identified during the processing of segment 124A by the track separation module 110 and / or during playback of vocal track 126A. Similar to segment 124A, segment 124B can comprise a single segment or multiple component segments. Segment 124B and segment 124A can be temporally adjacent and can at least partially overlap in time.
[0042] exist Figure 1C In the example, the track separation module 110 is activated to process segment 124B, such as... Figure 1CThe arrow extending from segment 124B to track separation module 110 indicates this. Processing of segment 124B by track separation module 110 can be initiated during processing of segment 124A by track separation module 110 and / or during playback of vocal track 126A. In some embodiments, track separation module 110 may include multiple instances of the same track separation module to allow simultaneous, parallel, or at least partially temporally overlapping processing of different segments (e.g., segments 124A and 124B) of the input audio signal (e.g., input audio signal 104). By processing segment 124B, track separation module 110 can generate additional track-separated segment 136B, which also includes audio tracks associated with various audio sources represented in segment 124B, including vocal track 126B, drum track 128B, bass track 130B, guitar track 132B, and other tracks 134B.
[0043] Figure 1C An arrow extending from vocal track 126B toward the output interface 106 of device 100 is shown, indicating that device 100 queues vocal track 126B for playback. Audio tracks queued for playback at consecutive points in time can correspond to the same audio source from input audio signal 104 (e.g., vocals, in...). Figure 1B , 1C (and 1D instances). Figure 1D The concept of the vocal track 126B being presented as having passed through the output interface 106 (traveling from left to right) is conceptually depicted. Figure 1C Playback of vocal track 126B. Vocal track 126B can be played back after vocal track 126A (or during the transition out of vocal track 126A). Various splicing or reconstruction processes can be performed on vocal tracks 126A and 126B to facilitate smooth transitions and continuous playback across the two tracks.
[0044] Figure 1D It also demonstrates the identification of another segment 124C from the input audio signal 104 and the processing of segment 124C by the track separation module 110 to obtain another track-separated segment 136C associated with multiple audio tracks (i.e., vocal track 126C, drum track 128C, bass track 130C, guitar track 132C, and other tracks 134C), which can be performed during the processing of segment 124B by the track separation module 110 or during playback of vocal track 126B (or any preceding track, such as vocal track 126A).
[0045] When the track splitting mode is enabled (indicated by box 118), the device 100 can continue to iteratively identify segments from the input audio signal 104, process the identified audio segments using the track splitting module 110 to obtain track-split segments, and cause playback of one or more audio tracks from the track-split segments until a stop condition is met (indicated by "Yes" extending from box 120). For a given iteration generating one or more track-split segments, the steps of identifying audio segments from the input audio signal 104 and / or processing audio segments using the track splitting module 110 to generate track-split segments occur during the processing of previous audio segments to generate previous track-split segments and / or during playback of one or more previous audio tracks from the previous track-split segments.
[0046] After the stopping condition is met, device 100 can deactivate the track separation mode (indicated by box 122) and continue to provide output audio signal 108 from input audio signal 104, as referenced above. Figure 1A The description (or the ability to stop / pause playback or perform different operations such as scrubbing / navigation, etc.) is provided.
[0047] In some implementations, the input audio signal 104 is associated with the input video signal. In such instances, when the track separation mode is "on," the playback of the input video signal can be selectively delayed to facilitate time synchronization with the playback of audio tracks separated from the input audio signal 104 by the track separation module 110. For example, the processing time for generating the track-separated segments by the track separation module can be determined by the system and used to delay the system's playback of video frames such that the playback of the video frames is time-synchronized with the playback of one or more audio tracks from the generated track-separated segments. The processing time can be predetermined and / or dynamically measured / updated. In some instances, the system utilizes the processing time in combination with a delay associated with a playback device (e.g., a wireless speaker) to synchronize the playback of video frames with the playback of the time-corresponding audio tracks (from the generated track-separated segments) on the playback device. The processing time associated with the track separation module and / or the latency of the playback device can be used to synchronize the timestamps of the video frames and audio tracks for playback (e.g., on the display and audio playback device).
[0048] Despite the above reference Figures 1A to 1DThe examples discussed focus, in at least some aspects, on implementations in which a track-separation mode facilitates playback of a single audio track (e.g., a vocal track) from the input audio signal 104. However, other playback configurations utilizing the separated tracks can be achieved by implementing the disclosed principles. For example, volume level changes or other transformations can be applied to individual audio tracks of the track-separated segment for playback, which can improve the spatial audio experience or facilitate, for example, speech enhancement to improve the clarity and / or volume of dialogue in audiovisual content. In this regard, one or more transformations or additional or alternative audio processing operations can be applied to one or more individual tracks of the track-separated segment for playback, and individual tracks that have been transformed or further processed (alone or in combination with one or more or all other tracks of the track-separated segment) can be played back.
[0049] Figure 2 and 3 Example flowcharts 200 and 300 are shown, respectively, depicting actions associated with the disclosed topic. (Reference) Figure 2 and 3 The described actions can be referenced below. Figure 12 The actions are performed by one or more components of the described system 1200, such as processor 1202, storage device 1204, sensor 1206, I / O system 1208, communication system 1210, remote system 1212, etc. Although the actions may be described and / or shown in a certain order, a specific order is not required unless explicitly stated or if the performance of one action depends on the completion of another action. It should be understood that one or more actions described herein may be omitted in the various embodiments.
[0050] Figure 2 Action 202 of flowchart 200 includes accessing an input audio signal. In some instances, the input audio signal comprises a digital audio signal associated with a digital audio file or digital audio transmission. In some embodiments, the input audio signal comprises a digital audio signal generated based on an input analog audio signal associated with an analog audio transmission. In some instances, the input audio signal is associated with an input video signal.
[0051] Action 204 of flowchart 200 includes identifying one or more segments of the first group of the input audio signal.
[0052] Action 206 of flowchart 200 includes using a track splitting module to process a first group of one or more segments of the input audio signal to generate a first group of one or more track-split segments, the first group of one or more track-split segments containing a first plurality of audio tracks corresponding to different audio sources represented in the first group of one or more segments.
[0053] Action 208 of flowchart 200 includes inducing playback of at least one audio track among a first plurality of audio tracks. In some embodiments, playback of at least one audio track among the first plurality of audio tracks includes avoiding playback of one or more remaining audio tracks among the first plurality of audio tracks. In some instances, inducing playback of at least one audio track among the first plurality of audio tracks includes transmitting a track-separated audio signal corresponding to at least one audio track among the first plurality of audio tracks to one or more playback devices. In some embodiments, inducing playback of at least one audio track among the first plurality of audio tracks includes inducing a speaker on one or more devices to play the track-separated audio signal corresponding to at least one audio track among the first plurality of audio tracks. In some instances, where the input audio signal is associated with an input video signal, playback of the input video signal can be delayed such that playback of the input video signal is time-synchronized with playback of at least one audio track among the first plurality of audio tracks and playback of at least one audio track among a second plurality of audio tracks.
[0054] Action 210 of flowchart 200 includes during the processing of a first group of one or more segments of an input audio signal using a track splitting module or during playback of at least one audio track among a first plurality of audio tracks: (i) identifying a second group of one or more segments of the input audio signal, wherein the first group of one or more segments and the second group of one or more segments of the input audio signal are temporally adjacent or temporally overlapping; and (ii) initiating processing of the second group of one or more segments of the input audio signal using the track splitting module to generate a second group of one or more track-split segments, the second group of one or more track-split segments containing a second plurality of audio tracks corresponding to different audio sources represented in the second group of one or more segments.
[0055] Action 212 of flowchart 200 includes, after the playback of at least one audio track among the first plurality of audio tracks, causing the playback of at least one audio track among the second plurality of audio tracks. In some embodiments, at least one audio track among the second plurality of audio tracks contains the same audio source as at least one audio track among the first plurality of audio tracks.
[0056] Figure 3 Action 302 of flowchart 300 includes inducing playback of the input audio signal. In some instances, the input audio signal comprises a digital audio signal associated with a digital audio file or digital audio transmission. In some embodiments, the input audio signal comprises a digital audio signal generated based on an input analog audio signal associated with an analog audio transmission.
[0057] Action 304 of flowchart 300 includes causing playback of the input audio signal to stop after detecting user input for activating the track splitting mode.
[0058] Action 306 of flowchart 300 includes, after detecting user input for activating the track splitting mode and until a stop condition is met, iteratively: (i) identifying a set of one or more segments of the input audio signal; (ii) using the track splitting module to process the set of one or more segments to generate a set of track-split segments, the set of track-split segments comprising multiple audio tracks corresponding to different audio sources represented in the set of one or more segments; and (iii) inducing playback from at least one audio track from the multiple audio tracks. In some instances, the track splitting module is selected based on: (i) user input selecting the track splitting module; or (ii) preprocessing the input audio signal to select the track splitting module. In some embodiments, the stop condition includes performing a predetermined number of iterations. In some instances, the stop condition includes detecting user input for deactivating the track splitting mode. In some implementations, playback of at least one audio track from the multiple audio tracks includes avoiding playback of one or more remaining audio tracks among the multiple audio tracks. In some instances, (i) a set of one or more segments of the input audio signal is identified, or (ii) a track splitting module is used to process a set of one or more segments to generate a set of track-split segments during: (a) processing a preceding set of one or more segments of the input audio signal using the track splitting module to generate a preceding set of track-split segments, the preceding set of track-split segments comprising multiple preceding audio tracks corresponding to different audio sources represented in the preceding set of one or more segments; or (b) playing back at least one preceding audio track from the multiple preceding audio tracks. In some embodiments, a preceding set of one or more segments is identified in a previous iteration after detecting user input for activating the track splitting mode.
[0059] Action 308 of flowchart 300 includes causing the playback of the input audio signal after the stop condition is met.
[0060] In some embodiments, at least some of the principles described herein may be implemented to facilitate system-level track splitting (or device-level track splitting) of system output audio signals (or device output audio signals). For example, the system or device may include an operating system (OS), which is a basic software layer that manages the system's hardware and software resources and provides an interface between users, applications, and hardware. Applications on the system or device contain programs designed to perform specific tasks for the user (e.g., media consumption, web browsing, word processing, communication, etc.). Applications executing on the system or device may generate audio signals or streams. For example, media players, video conferencing applications, games, or web browsers may generate sound and / or create their own independent audio signals based on the type of content they are processing. The system or device may include a system audio output signal path for synthesizing, mixing, and / or processing independent audio signals from various sound generation applications (and potentially from the OS itself) executing on the system or device to form the system output audio signal. The system audio output signal path may further send the system output audio signal to one or more playback devices to facilitate the playback of system-level sound / audio signals. The system audio output signal path may include various components for performing such functions, such as the OS audio subsystem (which may include various components such as audio routers, audio mixers, volume controllers, audio processors, digital-to-analog and / or analog-to-digital converters, driver managers, audio drivers, audio APIs, audio control panels, etc.), sound cards, audio chips, dedicated audio routing mechanisms / modules, etc.
[0061] In one example, to facilitate system-level track splitting functionality, the system can provide controls for activating or deactivating the system-level track splitting mode. When the system-level track splitting mode is active, the system can avoid sending or routing the system output audio signal to one or more playback devices via the system audio output signal path. Instead, the system can iteratively identify segments from the system output audio signal and use the track splitting module to process the identified segments to generate a track-split segment including audio tracks (e.g., similar to how segment 124A of the input audio signal 104 is processed by track splitting module 110 to provide a track-split segment 136A including audio tracks 126A, 128A, 130A, 132A, and 134A). One or more audio tracks from the audio tracks can then be sent to the playback device via the system audio output signal path.
[0062] In some implementations, system-level track splitting functionality can be controlled at least in part through one or more user interface front-ends. A user interface front-end can include visual or interactive portions of a program or website with which a user can interact. For example, a user interface front-end can include one or more sections, interfaces, pages, or components that can present information to the user (e.g., visually or otherwise) and / or provide a framework for facilitating user interaction, such as by receiving user input (e.g., providing input fields, optional elements / controls / buttons, etc.).
[0063] Figures 4 to 11 This demonstrates a sample user interface front-end that can facilitate or convey information related to system-level track separation functionality.
[0064] Figure 4 A sample user interface front-end 400 is shown, displaying controls 402 for activating or deactivating system-level track splitting mode. Figure 4 In the example shown, the user interface front-end 400 is presented on a display 410 associated with a desktop computer (e.g., a system 1200 of type 1200, as described below). In view of this disclosure, it should be understood that a user interface front-end including features and / or functions related to system-level track splitting as described herein can be presented on other types of devices / systems (e.g., smartphones, tablets, head-mounted displays, other wearable devices, etc.) and / or have different presentation characteristics (e.g., visual characteristics). Similarly, as previously described, the track splitting functionality described herein can be presented without reference to [reference missing]. Figures 4 to 11 In the case of a conceptually described user interface front-end, track separation can be implemented by hardware that performs this on a playback device such as a speaker or on an in-line device without a display.
[0065] In addition, Figure 4 In the example shown, the user interface front-end 400 is presented on the system through the execution of an application installed on the system. For example, Figure 4 An icon 404 is shown on the system / device for executing an application associated with the presentation of the user interface front-end 400. In view of this disclosure, it should be understood that a user interface front-end including features and / or functions related to system-level splitting as described herein can be part of the system's OS (e.g., part of the OS layer rather than the system's application layer). For example, Figure 4Additionally, a control element 406 is shown, which, when selected, can display OS-level volume / audio controls associated with the OS layer (e.g., master volume, application-specific volume mixer, etc.). In some embodiments, the selection of an OS-level control element for configuring the audio / volume aspects of the device / system can trigger the presentation of an OS-level user interface front-end, which facilitates system-level track separation control (e.g., presentation of functions and / or information similar to the user interface front-end 400 described herein). In this regard, track separation and / or other modules for implementing system-level track separation can be within the application layer and / or OS layer of the system or device.
[0066] Figure 4 The control 402 for activating or deactivating the system-level track splitting mode is depicted as a toggle switch for turning the mode / setting on or off. In various embodiments, the controls on the user interface for activating and / or deactivating the system-level track splitting mode can take various forms. Figure 4 Control 402 is depicted in a disabled state, indicating that the system-level track splitting mode is off. When the system-level track splitting mode is off, the system-generated system output audio signal can be sent to one or more playback devices for playback. The system output audio signal may contain a synthesis or mix of audio signals associated with all sound generation applications running on the system, and may additionally include audio signals from the OS itself. The system output audio signal can be generated and / or sent to playback devices via the system's system audio output signal path, which may include the OS audio subsystem and / or other components.
[0067] For clarification, please refer to the above reference. Figure 1A In the example discussed, when the system-level track splitting mode is off, the system output audio signal can be treated as the input audio signal 104, which can be sent to one or more playback devices (e.g., sent to output interface 106 for playback as output audio signal 108) through the system audio output signal path, without being processed by the track splitting module 110.
[0068] Figure 5A user interface front-end 400 is shown, in which control 402 is active, indicating that the system-level track splitting mode is enabled. Activation / deactivation of the system-level track splitting mode can be achieved through user input pointing to control 402 (e.g., touch, click, voice, gesture, and / or other types of user input). When the system-level track splitting mode is active, the system-generated system output audio signal can be rerouted to the track splitting pipeline. For example, the system can avoid sending the system output audio signal to a playback device, instead iteratively identifying segments from the system output audio signal and processing those segments using a selected track splitting module. The selected track splitting module can generate track-split segments, each of which can include audio tracks associated with different audio sources (e.g., vocals, drums, bass, guitar, and other / remaining audio or other sets of audio sources). One or more audio tracks represented in the track-split segments generated by the selected track splitting module can be sent to a playback device for playback (e.g., via the system audio output signal path). In some implementations, system-level track splitting occurs in real-time or near real-time. For example, the selected track splitting module can process segments identified from the system output audio signal while simultaneously processing one or more temporally earlier segments from the system output audio signal (e.g., in parallel) and / or simultaneously sending or playing back one or more audio tracks from previously generated track-split segments to a playback device. As another example, after the system output audio signal is generated by the system (or after the system output audio signal is provided to the track splitting module), one or more track-split segments can be generated by the track splitting module processing the system output audio signal (or segments identified therefrom) within a time frame of less than 1 second, less than 900 milliseconds, less than 800 milliseconds, less than 700 milliseconds, less than 600 milliseconds, less than 500 milliseconds, less than 400 milliseconds, less than 300 milliseconds, less than 200 milliseconds, less than 175 milliseconds, less than 150 milliseconds, less than 125 milliseconds, less than 100 milliseconds, less than 75 milliseconds, or less than 50 milliseconds. As yet another example, after the system generates the system output audio signal (or after the system output audio signal is provided to the track splitting module), one or more audio tracks from the track-split segment generated by the track splitting module can be sent to one or more playback devices for playback within a time of less than 1 second, less than 900 milliseconds, less than 800 milliseconds, less than 700 milliseconds, less than 600 milliseconds, less than 500 milliseconds, less than 400 milliseconds, less than 300 milliseconds, less than 200 milliseconds, less than 175 milliseconds, less than 150 milliseconds, less than 125 milliseconds, less than 100 milliseconds, less than 75 milliseconds, or less than 50 milliseconds.
[0069] For clarification, please refer to the above reference. Figures 1B to 1DIn the example discussed, when the system-level track splitting mode is enabled, the system output audio signal can be regarded as the input audio signal 104. From the input audio signal, segments (e.g., segments 124A, 124B, 124C) can be identified and processed by the track splitting module 110 to generate track-split segments (e.g., segments 136A, 136B, 136C) including audio tracks (e.g., tracks 126A, 128A, 130A, 132A, 134A, 126B, 128B, 130B, 132B, 134B, 126C, 128C, 130C, 132C, 134C). Any of these segments can be sent to a playback device (e.g., to output interface 106) for playback via the system audio output signal path. When the track separation mode is enabled, the system can avoid sending the system output audio signal itself to the playback device (e.g., to output interface 106) for playback via the system audio output signal path.
[0070] exist Figure 5 In the example shown, the user interface front-end 400 includes optional elements 502, 504, and 506, each associated with a different track separation module that can be used to process segments identified from the system output audio signal. For example, optional element 502 is associated with a music track separation module, optional element 504 with a movie or cinematic track separation module, and optional element 506 with a conference track separation module. The track separation modules associated with optional elements 502, 504, and 506 can be configured to generate track-separated segments that include a corresponding set of audio sources or tracks. Different track separation modules can have different sets of audio sources or tracks represented in their output. For example, in Figure 5 In the example shown, the music track splitting module associated with optional element 502 outputs track-split segments, including vocal tracks, drum tracks, bass tracks, guitar tracks, and other tracks (representing the remaining audio). In view of this disclosure, it should be understood that other audio sources / tracks (e.g., string tracks, piano tracks, keyboard instrument tracks, wind instrument tracks, and / or others) can be implemented for the music track splitting module.
[0071] In some implementations, the selected track separation module for processing segments identified from the system output audio signal is determined by user input to the user interface front end 400. For example, Figure 5 An example is shown where an optional element 502 associated with the music track splitting module is selected via a user interface front-end 400, instructing the music track splitting module to perform track splitting on segments identified from the system output audio signal. The selection of optional element 502 is... Figure 5 The text uses bolding and underlining to indicate meaning. Figure 5 An example of a volume mixing control 508 provided by the user interface front-end 400 is also shown. This volume mixing control includes audio track / source controls for different audio tracks / sources output by a music track splitting module (e.g., associated with optional element 502). For example, the volume mixing control 508 includes a vocal control 510, a drum control 512, a bass control 514, a guitar control 516, and other controls 518. Figure 5 In the example shown, controls for different audio tracks / sources are provided as slider controls, which control the contribution of each audio track / source to the final audio output that is then sent to a playback device (e.g., via the system audio output signal path). In this respect, the audio track (and / or its volume or contribution) that is sent to the playback device for playback can be managed by a volume mixer control provided at the front end 400 of the user interface. Other control modalities / forms are possible for the volume mixer control.
[0072] exist Figure 5 In the example shown, the volume mixing control 508 is configured to cause the full volume of each of the audio tracks / sources output by the music track splitting module to be provided to the playback device. Figure 6 An example is shown where the vocal control 510 is set to off (or the volume is completely reduced), which causes the vocal track output by the music track splitting module to be omitted from the final audio output sent by the system to the playback device. In other instances, the vocal control 510 can be set to have a reduced volume, which causes the vocal track output by the music track splitting module to be sent to the playback device at a reduced volume (e.g., relative to other tracks). Figure 7 An example is shown where the bass control 514 is set to off, which causes the bass track output by the music track splitting module to be omitted in the final audio output sent by the system to the playback device. The audio track output by the selected track splitting module can then be sent to the playback device according to the volume mixing controls provided on the user interface front end 400. Any volume mixing settings for providing the audio track output by the music track splitting module to the playback device can be selected and implemented via the volume mixing control 508.
[0073] Figure 8 An example is shown where an optional element 504 associated with the movie track splitting module is selected via a user interface front-end 400, instructing the movie track splitting module to perform track splitting on segments identified from the system output audio signal. The selection of optional element 504 is... Figure 8The text uses bolding and underlining to indicate this. When optional element 504 is selected, the user interface front-end 400 provides a volume mixing control 802, which includes audio track / source controls for different audio tracks / sources output by the movie track splitting module. For example, the volume mixing control 802 includes a dialog control 804, a music control 806, effects controls 808, and other controls 810. Figure 8 In the example shown, the volume mixing control 802 is configured to cause the full volume of each of the audio tracks / sources output by the movie track splitting module to be provided to the playback device. Figure 9 An instance is shown where the music control 806, effects control 808, and other controls 810 are set to off. This causes the dialogue track output by the movie track splitter module to be the only audio track / source sent by the system to the playback device. Any volume mixing settings for providing the audio track output by the movie track splitter module to the playback device can be selected and implemented via the volume mixing control 802.
[0074] Figure 10 An example is shown where an optional element 506 associated with the conference track splitting module is selected via a user interface front-end 400, instructing the conference track splitting module to perform track splitting on segments identified from the system output audio signal. The selection of optional element 506 is... Figure 10 The text uses bolding and underlining to indicate this. When optional element 506 is selected, the user interface front-end 400 provides a volume mixing control 1002, which includes audio track / source controls for different audio tracks / sources output by the conference track splitting module. For example, the volume mixing control 1002 includes a voice control 1004 and other controls 1006. Figure 10 In the example shown, the volume mixing control 1002 is configured to cause the full volume of each of the audio tracks / sources output by the conference track splitting module to be provided to the playback device. Figure 11 An example is shown where other controls 1006 are set to off, which can cause the vocal track output by the conference track splitter module to be the only audio track / source sent by the system to the playback device. Any volume mixing settings for providing the audio track output by the conference track splitter module to the playback device can be selected and implemented via the volume mixing control 1002.
[0075] Although reference Figure 4-11The examples shown and described focus, in at least some aspects, on the configuration of selecting a track splitting module for processing the system output audio signal (or segments thereof) based on user input provided through the user interface front end 400, but other configurations are also possible. For example, the track splitting module for processing the system output audio signal may be automatically selected based on analysis or preprocessing of the system output audio signal itself (e.g., to determine identification information and / or existing audio tracks) or based on the sound generation application that contributes audio to the system output audio signal. For example, the audio type (e.g., music, movie, conference, or others) may be determined based on analysis of the system output audio signal or based on the identity or action / operation of the application contributing to the system output audio signal. The track splitting module to be used for performing system-level track splitting may then be selected based on the determined audio type. As an illustrative example, when a video conferencing application generates an audio signal for the system, the system may automatically select a conference track splitting module to perform system-level track splitting when system-level track splitting mode is active. As another example, when a media player application generates audio signals for the system, the system can automatically select a music or movie track splitting module (based on the type of media played by the media player application) to perform system-level track splitting when system-level track splitting mode is active. The user interface front end can be used to configure volume mixing settings for playback of audio tracks / sources output by each of the different track splitting modules when automatically selected.
[0076] Figure 12 Example components of system 1200, which may include or implement aspects of one or more of the disclosed embodiments, are shown. For example, Figure 12 The implementation scheme of system 1200, including processor 1202, storage device 1204, sensor 1206, I / O system 1208, and communication system 1210, is shown. Although Figure 12 System 1200 is shown as including certain components, but in view of this disclosure it should be understood that system 1200 may contain any number of additional or alternative components.
[0077] Processor 1202 may include one or more sets of electronic circuit systems, comprising any number of logic units, registers, and / or control units to facilitate the execution of computer-readable instructions (e.g., instructions forming a computer program). Processor 1202 may take various forms, such as a CPU, NPU, GPU, or other types of processing units. Such computer-readable instructions may be stored within storage device 1204. Storage device 1204 may include physical system memory and may be volatile, non-volatile, or a combination thereof. Furthermore, storage device 1204 may include local storage, remote storage (e.g., accessible via communication system 1210 or other means), or a combination thereof. Additional details relating to the processor (e.g., processor 1202) and computer storage media (e.g., storage device 1204) will be provided below.
[0078] In some implementations, processor 1202 may include or be configured to execute any combination of software and / or hardware components operable to facilitate processing using machine learning models or other artificial intelligence-based architectures. For example, processor 1202 may include and / or utilize hardware components or computer-executable instructions operable to execute function blocks and / or processing layers, said function blocks and / or processing layers being configured to take the form of, as non-limiting examples, single-layer neural networks, feedforward neural networks, radial basis function networks, deep feedforward networks, recurrent neural networks, long short-term memory (LSTM) networks, gated recurrent units, autoencoder neural networks, variational autoencoders, denoising autoencoders, sparse autoencoders, Markov chains, Hopfield neural networks, Boltzmann machine networks, restricted Boltzmann machine networks, deep belief networks, deep convolutional networks (or convolutional neural networks), deconvolutional neural networks, deep convolutional inverse graph networks, Transformer networks, generative adversarial networks, liquid machines, extreme learning machines, echo state networks, deep residual networks, Kohonen networks, support vector machines, and neural Turing machines. machine), combinations thereof (or combinations of its components) and / or others.
[0079] As will be described in more detail, processor 1202 may be configured to execute instructions stored in storage device 1204 to perform certain actions. In some instances, the actions may depend at least in part on communication system 1210 for receiving data from remote system 1212, which may include, for example, a separate system or computing device, sensor, server, and / or others. Communication system 1210 may include any combination of software or hardware components operable to facilitate communication between components / devices on the system and / or with components / devices outside the system. For example, communication system 1210 may include ports, buses, or other physical connection devices for communicating with other devices / components. Additionally or alternatively, communication system 1210 may include systems / components operable to wirelessly communicate with external systems and / or devices via any suitable communication channel, as non-limiting examples such as Bluetooth, ultra-wideband, WLAN, infrared communication, and / or others.
[0080] Figure 12 The system 1200 is shown to include or communicate with a sensor 1206. The sensor 1206 may include any means for capturing or measuring data representing a perceptible phenomenon. As a non-limiting example, the sensor 1206 may include one or more image sensors, microphones, thermometers, barometers, magnetometers, accelerometers, gyroscopes, and / or others.
[0081] also, Figure 12 The system 1200 is shown to include or communicate with an I / O system 1208. The I / O system 1208 may include any type of input or output device, such as, but not limited to, a display, touch screen, mouse, keyboard, controller and / or other device as non-limiting examples.
[0082] The disclosed embodiments include at least those presented in the following numbered clauses: Clause 1. An apparatus comprising: one or more processing units; and one or more computer-readable recording media storing instructions executable by the one or more processing units to configure the apparatus to: access an input audio signal; identify a first group of one or more segments of the input audio signal; process the first group of one or more segments of the input audio signal using a track splitting module to generate a first group of one or more track-split segments, the first group of one or more track-split segments comprising a first plurality of audio tracks corresponding to different audio sources represented in the first group of one or more segments; induce playback of at least one of the first plurality of audio tracks; process the first group of one or more segments of the input audio signal using the track splitting module. During the playback of one or more audio tracks in the first plurality of audio tracks or during the playback of multiple segments: identifying a second group of one or more segments of the input audio signal, wherein the first group of one or more segments and the second group of one or more segments of the input audio signal are temporally adjacent or temporally overlapping; and using the track separation module to initiate processing of the second group of one or more segments of the input audio signal to generate a second group of one or more track-separated segments, the second group of one or more track-separated segments containing a second plurality of audio tracks corresponding to different audio sources represented in the second group of one or more segments; and after the playback of at least one audio track in the first plurality of audio tracks, causing the playback of at least one audio track in the second plurality of audio tracks.
[0083] Clause 2. The apparatus according to Clause 1, wherein the input audio signal comprises a digital audio signal associated with a digital audio file or digital audio transmission.
[0084] Clause 3. The apparatus according to Clause 1 or Clause 2, wherein the input audio signal comprises a digital audio signal generated based on an input analog audio signal associated with analog audio transmission.
[0085] Clause 4. The apparatus according to any one of Clauses 1 to 3, wherein playback of at least one audio track among the first plurality of audio tracks includes avoiding playback of one or more remaining audio tracks among the first plurality of audio tracks.
[0086] Clause 5. The apparatus according to any one of Clauses 1 to 4, wherein causing playback of at least one of the first plurality of audio tracks comprises transmitting a track-separated audio signal corresponding to at least one of the first plurality of audio tracks to one or more playback devices.
[0087] Clause 6. An apparatus according to any one of Clauses 1 to 5, wherein the apparatus comprises one or more speakers, and wherein causing playback of at least one of the first plurality of audio tracks comprises causing the one or more speakers to play a track-separated audio signal corresponding to the at least one of the first plurality of audio tracks.
[0088] Clause 7. The apparatus according to any one of Clauses 1 to 6, wherein at least one of the second plurality of audio tracks contains the same audio source as at least one of the first plurality of audio tracks.
[0089] Clause 8. The apparatus according to any one of Clauses 1 to 7, wherein the input audio signal is associated with an input video signal.
[0090] Clause 9. The apparatus according to Clause 8, wherein the instructions are executable by the one or more processing units to configure the apparatus to: cause a delayed playback of the input video signal such that the playback of the input video signal is time-synchronized with the playback of at least one audio track in the first plurality of audio tracks and with the playback of at least one audio track in the second plurality of audio tracks.
[0091] Clause 10. An apparatus comprising: one or more processing units; and one or more computer-readable recording media storing instructions executable by the one or more processing units to configure the apparatus to: cause playback of an input audio signal; and, upon detecting user input for activating a track splitting mode: cause playback of the input audio signal to stop; and, iteratively, until a stop condition is met, identify a set of one or more segments of the input audio signal; process the set of one or more segments using a track splitting module to generate a set of track split segments, the set of track split segments comprising a plurality of audio tracks corresponding to different audio sources represented in the set of one or more segments; and cause playback of at least one audio track from the plurality of audio tracks.
[0092] Clause 11. The apparatus according to Clause 10, wherein the input audio signal comprises a digital audio signal associated with a digital audio file or digital audio transmission.
[0093] Clause 12. The apparatus according to Clause 10 or Clause 11, wherein the input audio signal comprises a digital audio signal generated based on an input analog audio signal associated with analog audio transmission.
[0094] Clause 13. The apparatus according to any one of Clauses 10 to 12, wherein the stopping condition comprises performing a predetermined number of iterations.
[0095] Clause 14. The apparatus according to any one of Clauses 10 to 13, wherein the stopping condition includes detecting user input for disabling the track separation mode.
[0096] Clause 15. The apparatus according to any one of Clauses 10 to 14, wherein the instructions are executable by the one or more processing units to configure the apparatus to cause playback of the input audio signal after the stop condition is met.
[0097] Clause 16. The apparatus according to any one of Clauses 10 to 15, wherein playback of at least one audio track from the plurality of audio tracks includes avoiding playback of one or more remaining audio tracks among the plurality of audio tracks.
[0098] Clause 17. An apparatus according to any one of Clauses 10 to 16, wherein the instructions are executable by the one or more processing units to configure the apparatus to: (i) identify the set of one or more segments of the input audio signal; or (ii) use the track splitting module to process the set of one or more segments to generate the set of track-split segments during: (a) using the track splitting module to process a preceding set of one or more segments of the input audio signal to generate a preceding set of track-split segments, the preceding set of track-split segments comprising a plurality of preceding audio tracks corresponding to different audio sources represented in the preceding set of one or more segments; or (b) playing back at least one preceding audio track from the plurality of preceding audio tracks.
[0099] Clause 18. The apparatus according to Clause 17, wherein, in a previous iteration, after detecting user input for activating the track splitting mode, the preceding set of one or more segments is identified.
[0100] Clause 19. The apparatus according to any one of Clauses 10 to 18, wherein the track splitting module is selected based on: (i) user input to select the track splitting module; or (ii) preprocessing the input audio signal to select the track splitting module.
[0101] Clause 20. A method comprising: accessing an input audio signal; identifying a first group of one or more segments of the input audio signal; processing the first group of one or more segments of the input audio signal using a track splitting module to generate a first group of one or more track-split segments, the first group of one or more track-split segments comprising a first plurality of audio tracks corresponding to different audio sources represented in the first group of one or more segments; inducing playback of at least one of the first plurality of audio tracks; during processing the first group of one or more segments of the input audio signal using the track splitting module or during playback of the at least one of the first plurality of audio tracks: identifying The input audio signal is further divided into a second group of one or more segments, wherein the first group of one or more segments and the second group of one or more segments are temporally adjacent or temporally overlapping; and the processing of the second group of one or more segments of the input audio signal is initiated using the track splitting module to generate a second group of one or more track-split segments, the second group of one or more track-split segments containing a second plurality of audio tracks corresponding to different audio sources represented in the second group of one or more segments; and after the playback of at least one audio track in the first plurality of audio tracks, the playback of at least one audio track in the second plurality of audio tracks is caused.
[0102] Clause 21. A system comprising: a system audio output signal path configured to send a system output audio signal generated by the system to one or more playback devices; one or more processing units; and one or more computer-readable recording media storing instructions executable by the one or more processing units to configure the system to: present a user interface front end displaying controls for activating a system-level track splitting mode; and upon receiving user input for activating the controls, and when the system generates a system output audio signal based on audio associated with one or more applications executed on the system: avoid sending the system output audio signal to the one or more playback devices; identify a set of one or more segments of the system output audio signal; process the set of one or more segments using a selected track splitting module to generate a set of track-split segments comprising a plurality of audio tracks; and send at least one audio track from the plurality of audio tracks to the one or more playback devices via the system audio output signal path.
[0103] Clause 22. The system according to Clause 21, wherein the user interface front end is presented by the execution of an application installed on the system.
[0104] Clause 23. The system described in Clause 21 or Clause 22, wherein the user interface front end constitutes part of the operating system of the system.
[0105] Clause 24. The system according to any one of Clauses 21 to 23, wherein the system audio output signal path includes the operating system audio subsystem.
[0106] Clause 25. The system according to any one of Clauses 21 to 24, wherein the system output audio signal comprises a mix of audio from all sound generation applications performed on the system.
[0107] Clause 26. The system according to any one of Clauses 21 to 25, wherein the user interface front end provides a plurality of optional elements, wherein each of the plurality of optional elements is associated with a corresponding track splitting module from a plurality of track splitting modules, wherein each corresponding track splitting module is associated with a corresponding set of audio sources.
[0108] Clause 27. The system described in Clause 26, wherein at least two corresponding track separation modules are associated with different groups of audio sources.
[0109] Clause 28. The system according to Clause 26 or Clause 27, wherein the selected track separation module is selected from the plurality of track separation modules based on user input selecting one of the plurality of optional elements.
[0110] Clause 29. The system according to any one of Clauses 26 to 28, wherein the selected track splitting module is selected from the plurality of track splitting modules based on the system output audio signal or based on the one or more applications executed on the system.
[0111] Clause 30. The system according to any one of Clauses 26 to 29, wherein the user interface front end provides each of the plurality of optional elements with a corresponding set of volume mixing controls for the corresponding set of audio sources associated with the corresponding track splitting module.
[0112] Clause 31. The system according to Clause 30, wherein at least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path according to volume mixing settings defined by the corresponding set of volume mixing controls associated with the selected track separation module.
[0113] Clause 32. A system according to any one of Clauses 21 to 31, wherein the use of the selected track splitting module to process the set of one or more segments to generate the set of track-split segments is performed during: (a) using the selected track splitting module to process the preceding set of one or more segments of the system output audio signal to generate the preceding set of track-split segments comprising a plurality of preceding audio tracks; or (b) sending at least one preceding audio track from the plurality of preceding audio tracks to the one or more playback devices via the system audio output signal path.
[0114] Clause 33. A system comprising: a system audio output signal path configurable to send a system output audio signal generated by the system to one or more playback devices; one or more processing units; and one or more computer-readable recording media storing instructions executable by the one or more processing units to configure the system to: present a user interface front end displaying controls for activating a system-level track splitting mode, wherein the user interface front end provides a plurality of optional elements, each of the plurality of optional elements being associated with a corresponding track splitting module from a plurality of track splitting modules, wherein each corresponding track splitting module is associated with a corresponding set of audio sources, wherein the user interface front end provides each of the plurality of optional elements with a means for interacting with the corresponding track splitting module. A corresponding set of volume mixing controls associated with the corresponding set of audio sources; and upon receiving user input for the controls used to activate the system-level track splitting mode, and when the system generates a system output audio signal based on audio associated with one or more applications running on the system: avoiding sending the system output audio signal to the one or more playback devices; identifying a set of one or more segments of the system output audio signal; processing the set of one or more segments using a selected track splitting module from the plurality of track splitting modules to generate a set of track split segments, the set of track split segments comprising multiple audio tracks corresponding to the corresponding set of audio sources associated with the selected track splitting module; and sending at least one audio track from the plurality of audio tracks to the one or more playback devices via the system audio output signal path.
[0115] Clause 34. The system according to Clause 33, wherein the user interface front end is presented by the execution of an application installed on the system.
[0116] Clause 35. The system described in Clause 33 or Clause 34, wherein the user interface front end constitutes part of the operating system of the system.
[0117] Clause 36. The system according to any one of Clauses 33 to 35, wherein the system audio output signal path includes the operating system audio subsystem.
[0118] Clause 37. The system according to any one of Clauses 33 to 36, wherein the system output audio signal comprises a mix of audio from all sound generation applications performed on the system.
[0119] Clause 38. A system pursuant to any one of Clauses 33 to 37, wherein at least two corresponding track separation modules are associated with different groups of audio sources.
[0120] Clause 39. The system according to any one of Clauses 33 to 38, wherein the selected track separation module is selected from the plurality of track separation modules based on user input selecting one of the plurality of optional elements.
[0121] Clause 40. The system according to any one of Clauses 33 to 39, wherein the selected track splitting module is selected from the plurality of track splitting modules based on the system output audio signal or based on the one or more applications executed on the system.
[0122] Clause 41. The system according to any one of Clauses 33 to 40, wherein at least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path according to volume mixing settings defined by the corresponding set of volume mixing controls associated with the selected track separation module.
[0123] Clause 42. A system according to any one of Clauses 33 to 41, wherein the use of the selected track splitting module to process the set of one or more segments to generate the set of track-split segments is performed during: (a) using the selected track splitting module to process the preceding set of one or more segments of the system output audio signal to generate the preceding set of track-split segments comprising a plurality of preceding audio tracks; or (b) sending at least one preceding audio track from the plurality of preceding audio tracks to the one or more playback devices via the system audio output signal path.
[0124] Clause 43. A method comprising: presenting a user interface front end displaying controls for activating a system-level track splitting mode; and upon receiving user input for the controls for activating the system-level track splitting mode, and when a system generates a system output audio signal based on audio associated with one or more applications executing on the system: avoiding sending the system output audio signal to one or more playback devices via a system audio output signal path of the system, the system audio output signal path being configurable to send the system output audio signal generated by the system to the one or more playback devices; identifying a set of one or more segments of the system output audio signal; processing the set of one or more segments using a selected track splitting module to generate a set of track-split segments, the set of track-split segments comprising a plurality of audio tracks; and sending at least one audio track from the plurality of audio tracks to the one or more playback devices via the system audio output signal path.
[0125] Clause 44. The method according to Clause 43, wherein the user interface front end is presented by the execution of an application installed on the system.
[0126] Clause 45. The method described in accordance with Clause 43 or Clause 44, wherein the user interface front end constitutes part of the operating system of the system.
[0127] Clause 46. The method according to any one of Clauses 43 to 45, wherein the system audio output signal path includes the operating system audio subsystem.
[0128] Clause 47. The method according to any one of Clauses 43 to 46, wherein the system output audio signal comprises a mix of audio from all sound generation applications performed on the system.
[0129] Clause 48. The method according to any one of Clauses 43 to 47, wherein the user interface front end provides a plurality of optional elements, wherein each of the plurality of optional elements is associated with a corresponding track splitting module from a plurality of track splitting modules, wherein each corresponding track splitting module is associated with a corresponding set of audio sources.
[0130] Clause 49. The method described in Clause 48, wherein at least two corresponding track separation modules are associated with different groups of audio sources.
[0131] Clause 50. The method according to Clause 48 or Clause 49, wherein the selected track separation module is selected from the plurality of track separation modules based on user input selecting one of the plurality of optional elements.
[0132] Clause 51. The method according to any one of Clauses 48 to 50, wherein the selected track splitting module is selected from the plurality of track splitting modules based on the system output audio signal or based on the one or more applications performed on the system.
[0133] Clause 52. The method according to any one of Clauses 48 to 51, wherein the user interface front end provides each of the plurality of optional elements with a corresponding set of volume mixing controls for the corresponding set of audio sources associated with the corresponding track splitting module.
[0134] Clause 53. The method according to Clause 52, wherein at least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path according to volume mixing settings defined by the corresponding set of volume mixing controls associated with the selected track separation module.
[0135] Clause 54. The method according to any one of Clauses 43 to 53, wherein the use of the selected track splitting module to process the set of one or more segments to generate the set of track-split segments is performed during: (a) using the selected track splitting module to process the preceding set of one or more segments of the system output audio signal to generate the preceding set of track-split segments comprising a plurality of preceding audio tracks; or (b) sending at least one preceding audio track from the plurality of preceding audio tracks to the one or more playback devices via the system audio output signal path.
[0136] The disclosed embodiments may include or utilize dedicated or general-purpose computers including computer hardware, as discussed in more detail below. The disclosed embodiments also include physical media and other computer-readable media for carrying or storing computer-executable instructions and / or data structures. Such computer-readable media may be any available media accessible by a general-purpose or dedicated computer system. A computer-readable medium storing computer-executable instructions in data form is one or more “physical computer storage media” or “computer-readable recording media” or “hardware storage devices.” A computer-readable medium that carries only computer-executable instructions and does not store computer-executable instructions is a “transmission medium.” Therefore, by way of example and not limitation, the present embodiments may include at least two distinctly different kinds of computer-readable media: computer storage media and transmission media.
[0137] Computer storage media (also known as “hardware storage devices”) are computer-readable hardware storage devices such as RAM, ROM, EEPROM, CD-ROM, RAM-based, flash memory, phase-change memory (“PCM”), or other types of memory, solid-state drives (“SSDs”), or other optical disc storage devices, disk storage devices, or other magnetic storage devices, or any other medium that can be used to store desired program code in hardware in the form of computer-executable instructions, data, or data structures and that can be accessed by a general-purpose or special-purpose computer.
[0138] A “network” is defined as one or more data links that enable the transmission of electronic data between computer systems and / or modules and / or other electronic devices. When information is transferred or provided to a computer via a network or other communication connection (hardwired, wireless, or a combination of hardwired and wireless), the computer appropriately regards the connection as a transmission medium. The transmission medium may include networks and / or data links that can be used to carry program code in the form of computer-executable instructions or data structures and can be accessed by a general-purpose or special-purpose computer. Combinations of the foregoing are also included within the scope of computer-readable media.
[0139] Furthermore, upon arrival at various computer system components, program code in the form of computer-executable instructions or data structures can be automatically transferred from the transmission computer-readable medium to the physical computer-readable storage medium (and vice versa). For example, computer-executable instructions or data structures received via a network or data link can be buffered in RAM within a network interface module (e.g., a "NIC") and then ultimately transferred to the computer system RAM and / or a less volatile computer-readable physical storage medium on the computer system. Therefore, a computer-readable physical storage medium can be included in computer system components that also (or even primarily) utilize the transmission medium.
[0140] Computer-executable instructions comprise, for example, instructions and data, which cause a general-purpose computer, special-purpose computer, or special-purpose processing device to perform a function or a set of functions. Computer-executable instructions can be, for example, binary, intermediate format instructions such as assembly language, or even source code. Although the subject matter has been described in language specific to structural features and / or methodological actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the features or actions described above. Rather, the described features and actions are disclosed as examples of the claims.
[0141] The disclosed embodiments may include or utilize cloud computing. The cloud model may consist of various characteristics (e.g., on-demand self-service, broad network access, resource pooling, rapid elasticity, metered services, etc.), service models (e.g., Software as a Service (“SaaS”), Platform as a Service (“PaaS”), Infrastructure as a Service (“IaaS”)), and deployment models (e.g., private cloud, community cloud, public cloud, hybrid cloud, etc.).
[0142] Those skilled in the art will understand that at least some aspects of the present invention can be practiced in network computing environments with many types of computer system configurations, including personal computers, desktop computers, laptop computers, message processors, handheld devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, network PCs, minicomputers, mainframe computers, mobile phones, PDAs, pagers, routers, switches, wearable devices, etc. The present invention can also be practiced in distributed system environments, where multiple computer systems (e.g., local and remote systems) linked via a network (via hardwired data links, wireless data links, or a combination of hardwired and wireless data links) perform tasks. In a distributed system environment, program modules can be located in local and / or remote memory storage devices.
[0143] Alternatively or additionally, at least some of the functions described herein may be performed at least in part by one or more hardware logic components. For example, but not limited to, illustrative types of hardware logic components that may be used include field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), system-on-a-chip (SoCs), complex programmable logic devices (CPLDs), central processing units (CPUs), graphics processing units (GPUs), and / or others.
[0144] As used herein, the terms “executable module,” “executable component,” “component,” “module,” or “engine” can refer to a hardware processing unit or a software object, routine, or method that can be executed on one or more computer systems. The various components, modules, engines, and services described herein can be implemented as objects or processors that execute on one or more computer systems (e.g., implemented as a single thread).
[0145] It should be understood that any feature or operation disclosed herein can be combined with any or a combination of other features and operations disclosed herein. Additionally, content or features in any of the accompanying drawings can be combined or used in conjunction with any content or features used in any of the other drawings. In this respect, the content disclosed in any drawing is not mutually exclusive, but can be combined with content from any of the other drawings.
[0146] The invention may be embodied in other specific forms without departing from its spirit or characteristics. The described embodiments are to be regarded in all respects as illustrative rather than restrictive. Therefore, the scope of the invention is indicated by the appended claims rather than by the foregoing description. All modifications falling within the meaning and scope of equivalents of the claims are to be included within the scope of the claims.
Claims
1. A system comprising: a system audio output signal path, the system audio output signal path configurable to send a system output audio signal generated by the system to one or more playback devices; one or more processing units; and one or more computer-readable recording media storing instructions executable by the one or more processing units to configure the system to: present a user interface front-end displaying a control for activating a system- level track separation mode; and after receiving user input for the control for activating the system-level track separation mode, and while the system generates a system output audio signal based on audio associated with one or more applications executing on the system: avoid sending the system output audio signal to the one or more playback devices; identify a set of one or more segments of the system output audio signal; process the set of one or more segments using a selected track separation module to generate a set of track-separated segments, the set of track-separated segments including a plurality of audio tracks; and send at least one audio track from the plurality of audio tracks to the one or more playback devices through the system audio output signal path.
2. The system of claim 1, wherein the user interface front-end is presented through execution of an application installed on the system.
3. The system of claim 1, wherein the user interface front-end constitutes part of an operating system of the system.
4. The system of claim 1, wherein the system audio output signal path includes an operating system audio subsystem.
5. The system of claim 1, wherein the system output audio signal includes a mix of audio from all sound producing applications executing on the system.
6. The system of claim 1, wherein the user interface front-end provides a plurality of selectable elements, wherein each of the plurality of selectable elements is associated with a respective track separation module from a plurality of track separation modules, wherein each respective track separation module is associated with a respective set of audio sources.
7. The system of claim 6, wherein the selected track separation module is selected from the plurality of track separation modules based on user input selecting one of the plurality of selectable elements.
8. The system of claim 6, wherein the selected track separation module is selected from the plurality of track separation modules based on the system output audio signal or based on the one or more applications executing on the system.
9. The system of claim 6, wherein the user interface front-end provides for each of the plurality of selectable elements a respective set of volume mix controls for the respective set of audio sources associated with the respective track separation module. 10. The system of claim 9, wherein at least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path according to volume mixing settings defined by the corresponding set of volume mixing controls associated with the selected track separation module.
11. The system of claim 1, wherein using the selected track splitting module to process the set of one or more segments to generate the set of track-split segments is performed during: (a) using the selected track splitting module to process the preceding set of one or more segments of the system output audio signal to generate the preceding set of track-split segments comprising a plurality of preceding audio tracks; or (b) sending at least one preceding audio track from the plurality of preceding audio tracks to the one or more playback devices via the system audio output signal path.
12. A system comprising: A system audio output signal path, which can be configured to send a system output audio signal generated by the system to one or more playback devices; One or more processing units; as well as One or more computer-readable recording media storing instructions executable by the one or more processing units to configure the system to: The user interface front end displays controls for activating the system-level track splitting mode. The user interface front end provides multiple optional elements, each of which is associated with a corresponding track splitting module from multiple track splitting modules. Each corresponding track splitting module is associated with a corresponding set of audio sources. The user interface front end provides each of the multiple optional elements with a corresponding set of volume mixing controls for the corresponding set of audio sources associated with the corresponding track splitting module. and Upon receiving user input for the control used to activate the system-level track splitting mode, and when the system generates a system output audio signal based on audio associated with one or more applications running on the system: Avoid sending the system's output audio signal to the one or more playback devices; Identify one or more segments of the audio signal output by the system; The selected track splitting module from the plurality of track splitting modules is used to process the set of one or more segments to generate a set of track split segments, the set of track split segments comprising multiple audio tracks corresponding to the corresponding set of audio sources associated with the selected track splitting module; and At least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path.
13. The system of claim 12, wherein the user interface front end is presented by the execution of an application installed on the system.
14. The system of claim 12, wherein the user interface front end constitutes part of the operating system of the system.
15. The system of claim 12, wherein the system output audio signal comprises a mix of audio from all sound generation applications performed on the system.
16. The system of claim 12, wherein the selected track separation module is selected from the plurality of track separation modules based on user input that selects one of the plurality of optional elements.
17. The system of claim 12, wherein the selected track splitting module is selected from the plurality of track splitting modules based on the system output audio signal or based on the one or more applications executed on the system.
18. The system of claim 12, wherein at least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path according to volume mixing settings defined by the corresponding set of volume mixing controls associated with the selected track separation module.
19. The system of claim 12, wherein using the selected track splitting module to process the set of one or more segments to generate the set of track-split segments is performed during: (a) using the selected track splitting module to process the preceding set of one or more segments of the system output audio signal to generate the preceding set of track-split segments comprising a plurality of preceding audio tracks; or (b) sending at least one preceding audio track from the plurality of preceding audio tracks to the one or more playback devices via the system audio output signal path.
20. A method comprising: presenting a user interface front end that displays a control for activating a system-level track separation mode; as well as Upon receiving user input for the control used to activate the system-level track splitting mode, and when the system generates a system output audio signal based on audio associated with one or more applications running on the system: Avoid sending the system output audio signal to one or more playback devices through the system audio output signal path of the system, the system audio output signal path being configurable to send the system output audio signal generated by the system to the one or more playback devices; Identify one or more segments of the audio signal output by the system; The selected track splitting module is used to process the set of one or more segments to generate a set of track-split segments, the set of track-split segments containing multiple audio tracks; as well as At least one audio track from the plurality of audio tracks is sent to the one or more playback devices via the system audio output signal path.
