A mixed audio track separation method, system, device, and storage medium

Abstract

The application relates to the technical field of audio signal processing and artificial intelligence, in particular to a mixed audio track separation method, system, device and storage medium. The mixed audio track separation method comprises the following steps: extracting time domain features of a mixed audio signal through adaptive receptive field convolution, extracting frequency domain features of the mixed audio signal through an adaptive frequency spectrum window, and obtaining output features; generating reinforced features based on the output features, a pre-constructed frequency domain mask of a target instrument to be separated and feature channel attention; and performing adaptive waveform reconstruction through a joint decoder based on the reinforced features, so as to obtain separated independent audio tracks. The application can improve the efficiency and accuracy of mixed audio track separation.

Description

Technical Field

[0001] This application relates to the field of audio signal processing and artificial intelligence, and in particular to a method, system, device and storage medium for separating mixed audio tracks. Background Technology

[0002] Polyphonic music source separation is a key technology in the field of audio processing. While existing mainstream technologies (such as Open-Unmix and Demucs based on the U-Net architecture) have made some progress, the following technical bottlenecks still exist:

[0003] Spectral smearing and phase distortion: The method based on short-time Fourier transform (STFT) is limited by the fixed window length and cannot simultaneously take into account time resolution (important for transient signals) and frequency resolution (important for continuous sounds), resulting in a "smearing" effect in the separated audio, and the SDR (signal distortion ratio) index is difficult to break through the 15dB bottleneck.

[0004] Confusion of similar instrument features: For instruments with similar acoustic features (such as violin and viola), existing purely data-driven models have difficulty extracting features with sufficient discriminative power, resulting in a high crosstalk rate in the separation results.

[0005] Loss of high-frequency details: The downsampling process of deep networks often filters out high-frequency details, making high-frequency transient signals such as drum sounds and cymbals blurry after reconstruction.

[0006] Low information utilization due to task fragmentation: Existing solutions usually treat separation and recognition as independent tasks, ignoring the guiding role of instrument category information in the separation task, resulting in insufficient targeting of feature extraction. Summary of the Invention

[0007] This application provides a method, system, device, and storage medium for separating mixed audio tracks to solve the problems mentioned in the background art.

[0008] Firstly, this application provides a method for separating mixed audio tracks, including:

[0009] The temporal features of the mixed audio signal are extracted by adaptive receptive field convolution, and the frequency features of the mixed audio signal are extracted by adaptive spectral windowing to obtain the output features;

[0010] Enhanced features are generated based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention.

[0011] Based on the enhancement features, adaptive waveform reconstruction is performed through a joint decoder to obtain the separated independent audio tracks;

[0012] The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy:

[0013]

[0014] in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies .

[0015] Furthermore, the adaptive spectrum window performs parallel computation of STFT features with multiple window lengths and performs weighted fusion through a learnable gating mechanism.

[0016] Furthermore, it also includes:

[0017] The frequency band distribution and chord structure are transformed into a learnable frequency domain mask tensor, which is then embedded in the intermediate layer of the neural network to participate in training and guide the network to focus on specific frequency bands.

[0018] Furthermore, the enhancement feature generation based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention includes:

[0019] Calculate the local energy gradient of the output features, generate an energy heatmap, and locate the active instrument components;

[0020] The frequency domain mask, energy heatmap and feature channel attention are multiplied and fused to generate an attention weight map;

[0021] The enhanced features are obtained by multiplying the output features with the attention weight map.

[0022] Furthermore, the step of multiplying and fusing the frequency domain mask, energy heatmap, and feature channel attention to generate an attention weight map includes:

[0023] Call the pre-built instrument frequency band embedding library to obtain the frequency domain mask of the corresponding target instrument. ;

[0024] The spatiotemporal distribution of the data is learned through a 1×1 convolutional layer. ;

[0025] An auxiliary chroma feature extraction branch is introduced to analyze the chord attributes of the current frame and generate a chord probability vector. ;

[0026] Attention weight map obtained by fusion calculation :

[0027]

[0028] in This indicates a splicing operation. This is the Sigmoid activation function.

[0029] Furthermore, it also includes:

[0030] Following the deep feature layer of the joint decoder, an auxiliary instrument classifier is introduced, which outputs the instrument category label.

[0031] Secondly, this application provides a hybrid audio track separation system, comprising:

[0032] The feature extraction module is used to extract the temporal features of the mixed audio signal through adaptive receptive field convolution and extract the frequency domain features of the mixed audio signal through adaptive spectral window to obtain the output features;

[0033] The enhancement generation module is used to generate enhancement features based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention.

[0034] The audio track separation module is used to perform adaptive waveform reconstruction based on enhancement features through a joint decoder to obtain the separated independent audio tracks.

[0035] The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy:

[0036]

[0037] in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies .

[0038] Thirdly, this application provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the mixed audio track separation method described above.

[0039] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the mixed audio track separation method described above.

[0040] The above-mentioned technical solution of this application has the following advantages:

[0041] The mixed audio track separation method provided in the first aspect of this application extracts the temporal features of the mixed audio signal through adaptive receptive field convolution and extracts the frequency domain features of the mixed audio signal through adaptive spectral window to obtain output features. Based on the output features, the pre-constructed frequency domain mask of the target instrument to be separated, and feature channel attention, enhancement features are generated. Based on the enhancement features, adaptive waveform reconstruction is performed through a joint decoder to obtain the separated independent audio tracks, which can improve the efficiency and accuracy of mixed audio track separation.

[0042] It is understood that the beneficial effects of the second, third, and fourth aspects mentioned above can be found in the relevant descriptions in the first aspect above, and will not be repeated here. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 A flowchart illustrating the overall process of the mixed audio track separation method provided in this application embodiment;

[0045] Figure 2 A flowchart of the multi-scale dilated convolution weighted fusion strategy provided in the embodiments of this application;

[0046] Figure 3 A flowchart of the MPAN process provided in this application embodiment;

[0047] Figure 4 This is a flowchart illustrating the use of the AdamW optimizer to update network parameters, as provided in an embodiment of this application. Detailed Implementation

[0048] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of this application with unnecessary detail.

[0049] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0050] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0051] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. "A plurality" means "two or more."

[0052] This application relates to a deep learning method for separating individual audio tracks (such as vocals, drums, bass, guitar, etc.) from mixed audio, applicable to scenarios such as copyright monitoring, intelligent music production, and immersive audio experiences.

[0053] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0054] This application provides a method for separating mixed audio tracks, the overall process of which is as follows: Figure 1 As shown, the process includes: extracting temporal features of the mixed audio signal through adaptive receptive field convolution, extracting frequency domain features of the mixed audio signal through adaptive spectral windowing to obtain output features; generating enhanced features based on the output features, a pre-constructed frequency domain mask of the target instrument to be separated, and feature channel attention; and performing adaptive waveform reconstruction based on the enhanced features through a joint decoder to obtain the separated independent audio tracks. The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy.

[0055]

[0056] in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies .

[0057] This application proposes a four-order closed-loop deep learning architecture of "perception-decision-execution-feedback", the core of which is:

[0058] Construct a "Music Prior Guided Attention Network" (MPAN): transform music theory knowledge (frequency distribution, chord structure) into learnable tensor masks, embed them into the intermediate layer of a neural network, and solve the blindness of pure data-driven models in distinguishing similar instruments.

[0059] Adaptive multi-scale feature capture mechanism: By using ARFC (Adaptive Receptive Field Convolution) and ASW (Adaptive Spectral Window), the receptive field and time-frequency resolution are dynamically adjusted during the feature extraction stage to solve the spectral smearing problem.

[0060] Gradient backpropagation mechanism for auxiliary supervision: An auxiliary classification head is introduced in the intermediate layer of the decoder, and the weight distribution of the attention layer is directly optimized by using the gradient of the classification task, so as to achieve task collaboration of "promoting separation through recognition".

[0061] In some embodiments, the adaptive spectral window computes STFT features of multiple window lengths in parallel and performs weighted fusion through a learnable gating mechanism.

[0062] In some embodiments, the method further includes: converting the frequency band distribution and chord structure into a learnable frequency domain mask tensor, embedding it into the intermediate layer of the neural network for training, and guiding the network to focus on a specific frequency band.

[0063] In some embodiments, the generation of enhanced features based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention includes: calculating the local energy gradient of the output features, generating an energy heatmap, and locating active musical instrument components; performing dot product fusion of the frequency domain mask, energy heatmap, and feature channel attention to generate an attention weight map; and multiplying the output features with the attention weight map to obtain enhanced features.

[0064] In some embodiments, the step of multiplying and fusing the frequency domain mask, energy heatmap, and feature channel attention to generate an attention weight map includes:

[0065] Call the pre-built instrument frequency band embedding library to obtain the frequency domain mask of the corresponding target instrument. ;

[0066] The spatiotemporal distribution of the data is learned through a 1×1 convolutional layer. ;

[0067] An auxiliary chroma feature extraction branch is introduced to analyze the chord attributes of the current frame and generate a chord probability vector. ;

[0068] Attention weight map obtained by fusion calculation :

[0069]

[0070] in This indicates a splicing operation. This is the Sigmoid activation function.

[0071] In some embodiments, the method further includes: after the deep feature layer of the joint decoder, introducing an auxiliary instrument classifier to output instrument category labels.

[0072] The mixed audio track separation method provided in this application embodiment, such as Figure 1 As shown, it mainly includes four processing stages:

[0073] 1. Feature perception stage (dual-path extraction)

[0074] To capture signal features without loss, a parallel time-domain and frequency-domain dual-path network is employed:

[0075] Temporal path: The original input waveform is processed through the ARFC (Adaptive Receptive Field Convolution) module. This module does not directly change the physical convolution kernel size, but instead uses multi-branch parallel convolution combined with attention weighting to dynamically synthesize an effective receptive field to adapt to the temporal characteristics of different instruments.

[0076] Frequency domain path: Input spectrogram, processed through the ASW (Adaptive Spectrum Window) module. This module computes STFT features of various window lengths (e.g., 1024 / 2048 / 4096 points) in parallel and performs weighted fusion through a learnable gating mechanism, taking into account both time and frequency resolution.

[0077] 2. Attention Decision-Making Stage (MPAN Core)

[0078] MPAN is the core decision-making module in the embodiments of this application. It is no longer a "black box" but has a pre-set acoustic prior:

[0079] Frequency band prior embedding: Constructing a frequency mask tensor. For example, for "violin", the system initializes a mask vector with a response value of 1 in the 250Hz-1500Hz range and 0 elsewhere. This vector is used as a learnable parameter in training to guide the network to focus on specific frequency bands.

[0080] Dynamic Energy Focusing: Calculates the local energy gradient of the output features, generates a two-dimensional heatmap, and locates active instrument components.

[0081] Multidimensional fusion: The frequency domain mask, energy heatmap, and feature channel attention are fused by dot product to generate the final attention weight map. .

[0082] The final multi-dimensional attention weight map has the same dimensions as the output feature tensor X (C×T×F) and is used to weight and enhance the input features.

[0083] 3. Task Execution Phase (Joint Decoding)

[0084] Feature weighting: combining the output of the feature extraction layer with... Multiplying them together yields the enhanced feature F. fused .

[0085] Adaptive waveform reconstruction: The decoder dynamically selects a reconstruction strategy based on the transient strength of the currently processed signal. For strong transient signals (drums), the decoding weight of the time-domain path is increased; for steady-state signals (strings), the decoding weight of the frequency-domain path is increased.

[0086] 4. Feedback and Optimization Phase (Assisted Supervision)

[0087] Following the deep feature layer (Bottleneck) of the decoder, an auxiliary instrument classifier is introduced.

[0088] During training, the gradient flow generated by the classification loss (Cross-Entropy) does not pass through subsequent reconstruction layers, but is directly backpropagated to the MPAN layer. This forces the attention weights learned by MPAN to contain sufficient semantic information to distinguish instrument types, thus solving the problem of confusion between similar instruments at the source.

[0089] Key Module Implementation Principles

[0090] Specific implementation of Adaptive Receptive Field Convolution (ARFC)

[0091] To implement a variable receptive field in a computer, embodiments of this application employ a multi-scale dilated convolution weighted fusion strategy, the process of which is as follows: Figure 2 As shown: Formula model:

[0092] in:

[0093] Indicates the kernel size as Dilation Rate is Convolution operations (e.g.) ).

[0094] The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer (SE-Block structure) and satisfies .

[0095] Technical effect: When a transient signal (such as drumbeat) is detected, the network automatically increases. The weights are adjusted to capture details; when a long-term signal (such as bass) is detected, the weights are increased. The weight is used to expand the receptive field.

[0096] Music-Prior Guidance Attention Mechanism (MPAN)

[0097] The workflow of MPAN is as follows Figure 3 As shown, it includes the following steps:

[0098] Input: Feature tensor .

[0099] Prior branch: Call the pre-built instrument embedding bank to obtain the frequency domain mask of the corresponding target instrument. .

[0100] Data-driven branch: Learning the spatiotemporal distribution of the data itself through 1×1 convolutional layers. .

[0101] Chord context: An auxiliary chroma feature extraction branch is introduced to analyze the chord attributes of the current frame and generate a chord probability vector. .

[0102] Fusion computing:

[0103]

[0104] in This indicates a splicing operation. This is the Sigmoid activation function.

[0105] This method can be run on electronic devices that contain the following components:

[0106] Processor: High-performance GPU (such as NVIDIA A100 / RTX 4090) for parallel tensor operations.

[0107] Memory: Stores the pre-trained frequency band embedding library and model weights.

[0108] Audio interface: Supports high-resolution audio input and output of 44.1kHz / 24bit and above.

[0109] Training Process Implementation Examples

[0110] Data preparation: Using the MUSDB18-HQ dataset, random slicing (3-second segments) and data augmentation (pitch shift, mixing gain adjustment) were performed.

[0111] Network initialization: Initialize the convolutional kernel weights of ARFC and initialize the frequency band mask tensor in MPAN according to music theory (e.g., initialize the mask of the Bass track to low frequency high response).

[0112] Forward propagation:

[0113] a. The mixed audio is processed by ARFC to extract time-domain features and ASW to extract frequency-domain features.

[0114] b. MPAN combines real-time features with prior masks to generate attention maps.

[0115] c. The decoder outputs the separated waveform and the probability of the instrument category.

[0116] Loss calculation and backpropagation:

[0117] a. Construct the joint loss function:

[0118]

[0119] b. Update network parameters using the AdamW optimizer, where The gradient is used to update the parameters of the MPAN layer, such as Figure 4 As shown.

[0120] The beneficial effects of the hybrid audio track separation method provided in this application are reflected in the following aspects:

[0121] Significantly improved separation accuracy: The contradiction between time and frequency resolution was resolved by ARFC and ASW, and the SDR index was improved by about 29.5% compared with Open-Unmix (based on MUSDB18 test).

[0122] Strong ability to distinguish similar instruments: By using music theory priors (frequency domain mask) to guide attention, the crosstalk rate of similar instruments such as violin and viola is reduced by 48.5%.

[0123] High-frequency detail preservation: The adaptive waveform reconstruction strategy effectively preserves the transient details of drum sounds and cymbals, improving the MUSHRA subjective clarity score from 3.1 to 4.2.

[0124] Faster model training convergence: Gradient feedback provided by the auxiliary classification task enables the model to quickly lock key feature regions in the early stages of training, improving training efficiency by about 30%.

[0125] Corresponding to the exception-driven database multi-component diagnosis and tuning method described in the above embodiments, this application also provides an exception-driven database multi-component diagnosis and tuning system, which includes:

[0126] The feature extraction module is used to extract the temporal features of the mixed audio signal through adaptive receptive field convolution and extract the frequency domain features of the mixed audio signal through adaptive spectral window to obtain the output features;

[0127] The enhancement generation module is used to generate enhancement features based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention.

[0128] The audio track separation module is used to perform adaptive waveform reconstruction based on enhancement features through a joint decoder to obtain the separated independent audio tracks.

[0129] The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy:

[0130]

[0131] in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies .

[0132] It should be noted that the information interaction and execution process between the above modules / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0133] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above device can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0134] This application also provides a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the mixed audio track separation method provided in the first aspect.

[0135] In applications, terminal devices may include, but are not limited to, processors and memory. These are merely examples of terminal devices and do not constitute a limitation on them. They may include more or fewer components, combinations of certain components, or different components, such as input / output devices and network access devices. Input / output devices may include cameras, audio capture / playback devices, displays, etc. Network access devices may include network modules for wireless network communication with external devices.

[0136] In applications, the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor.

[0137] In applications, the memory may be an internal storage unit of the terminal device in some embodiments, such as the hard drive or RAM of the terminal device. In other embodiments, the memory may be an external storage device of the terminal device, such as a plug-in hard drive, a smart media card (SMC), or a flash card. The memory may also include both internal and external storage units of the terminal device. The memory is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of a computer program. The memory can also be used to temporarily store data that has been output or will be output.

[0138] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the steps in the above-described method embodiments.

[0139] This application implements all or part of the processes in the methods of the above embodiments, which can be accomplished by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium can include at least: any entity or device capable of carrying the computer program code to a terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, such as a USB flash drive, a portable hard drive, a magnetic disk, or an optical disk.

[0140] Those skilled in the art will recognize that the device and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0141] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interface, or the device may be indirectly coupled or communicated, and may be electrical, mechanical, or other forms.

[0142] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A method for separating mixed audio tracks, characterized in that, include: The temporal features of the mixed audio signal are extracted by adaptive receptive field convolution, and the frequency features of the mixed audio signal are extracted by adaptive spectral window. The temporal and frequency features are then concatenated to obtain the output features. Enhanced features are generated based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention. Based on the enhancement features, adaptive waveform reconstruction is performed through a joint decoder to obtain the separated independent audio tracks; The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy: in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies ; The enhancement features generated based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention include: Calculate the local energy gradient of the output features, generate an energy heatmap, and locate the active instrument components; The frequency domain mask, energy heatmap and feature channel attention are multiplied and fused to generate an attention weight map; The output features are multiplied by the attention weight map to obtain the enhanced features; The step of multiplying and fusing the frequency domain mask, energy heatmap, and feature channel attention to generate an attention weight map includes: Call the pre-built instrument frequency band embedding library to obtain the frequency domain mask of the corresponding target instrument. ; The spatiotemporal distribution of the data is learned through a 1×1 convolutional layer. ; An auxiliary chroma feature extraction branch is introduced to analyze the chord attributes of the current frame and generate a chord probability vector. ; Attention weight map obtained by fusion calculation : in This indicates a splicing operation. This is the Sigmoid activation function.

2. The method for separating mixed audio tracks as described in claim 1, characterized in that, The adaptive spectral window performs parallel computation of STFT features with multiple window lengths and performs weighted fusion through a learnable gating mechanism.

3. The method for separating mixed audio tracks as described in claim 1, characterized in that, Also includes: The frequency band distribution and chord structure are transformed into a learnable frequency domain mask tensor, which is then embedded in the intermediate layer of the neural network to participate in training and guide the network to focus on specific frequency bands.

4. The method for separating mixed audio tracks as described in claim 1, characterized in that, Also includes: Following the deep feature layer of the joint decoder, an auxiliary instrument classifier is introduced, which outputs the instrument category label.

5. A hybrid audio track separation system, characterized in that, include: The feature extraction module is used to extract the temporal features of the mixed audio signal through adaptive receptive field convolution, extract the frequency domain features of the mixed audio signal through adaptive spectral window, and concatenate the temporal and frequency domain features to obtain the output features. The enhancement generation module is used to generate enhancement features based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention. The audio track separation module is used to perform adaptive waveform reconstruction based on enhancement features through a joint decoder to obtain the separated independent audio tracks. The adaptive receptive field convolution employs a multi-scale dilated convolution weighted fusion strategy: in Indicates the kernel size as void ratio Convolution operation, The adaptive weighting coefficients are derived from the local energy of the input signal. It is calculated using a lightweight fully connected layer and satisfies ; The enhancement features generated based on output features, a pre-constructed frequency domain mask of the target musical instrument to be separated, and feature channel attention include: Calculate the local energy gradient of the output features, generate an energy heatmap, and locate the active instrument components; The frequency domain mask, energy heatmap and feature channel attention are multiplied and fused to generate an attention weight map; The output features are multiplied by the attention weight map to obtain the enhanced features; The step of multiplying and fusing the frequency domain mask, energy heatmap, and feature channel attention to generate an attention weight map includes: Call the pre-built instrument frequency band embedding library to obtain the frequency domain mask of the corresponding target instrument. ; The spatiotemporal distribution of the data is learned through a 1×1 convolutional layer. ; An auxiliary chroma feature extraction branch is introduced to analyze the chord attributes of the current frame and generate a chord probability vector. ; Attention weight map obtained by fusion calculation : in This indicates a splicing operation. This is the Sigmoid activation function.

6. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the mixed audio track separation method as described in any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the mixed audio track separation method as described in any one of claims 1 to 4.

Images