Enabling automatic music audio transcription

Through the combination of convolutional model and deep neural network model, combined with spectrum cross attention and time transformer, the problem of insufficient model scalability and instrument distinction ability in existing systems is solved, and the accuracy of multiple instruments and vocals is achieved. Generation of transcription and synthetic audio.

CN119068849APending Publication Date: 2024-12-03FACE CUTE CO LTD
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202410686524.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2023-06-01
Filing Date
2024-05-30
Publication Date
2024-12-03

AI Technical Summary

Technical Problem

The existing automatic music transcription system lacks model scalability and difficulty in distinguishing different instruments and vocals, resulting in inaccurate transcription.

Method used

Using convolutional model and deep neural network model, combining spectrum cross attention, potential transformer and time transformer, enhance training data through random mixing, improve the scalability and distinction ability of the model, and generate MIDI output to transcribe multiple instruments and humans. Voice.

Benefits of technology

The computational scalability and instrument discernment capabilities of the model are significantly improved, and the ability to accurately transcribe multiple instruments and vocals are able to generate synthetic audio similar to the original audio.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119068849A_ABST
    Figure CN119068849A_ABST
Patent Text Reader

Abstract

This disclosure describes techniques for implementing automatic music audio transcription. The deep neural network model may be configured. The deep neural network model includes a spectral cross attention sub-model configured to project a spectral representation (represented as St) of each time step t into a set of potential arrays (represented as [theta] th) at the time step t, h representing the hth iteration. The deep neural network model includes a plurality of potential converters configured to perform self-attention on a set of potential arrays [theta] th. The deep neural network model further comprises a set of time transducers configured to enable communication between any pair of potential arrays [theta] th at different time steps. Training data may be enhanced by randomly mixing multiple types of datasets, including a voice dataset and a musical instrument dataset. The deep neural network model may be trained using the enhanced training data.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] Music information retrieval involves retrieving information from music. Music information retrieval tasks are widely used in the music, filmmaking, social media, and entertainment industries. Such tasks may include multi-pitch detection, onset detection, duration estimation, instrument identification, and extraction of harmonic, rhythmic, or melodic information. However, due to various limitations, conventional music information retrieval techniques may not meet the needs of users. Therefore, there is a need to improve music information retrieval. BRIEF DESCRIPTION OF THE DRAWINGS

[0002] The following detailed description will be better understood when read in conjunction with the accompanying drawings.For purposes of illustration, example embodiments of various aspects of the disclosure are shown in the drawings; however, the invention is not limited to the specific methods and instrumentalities disclosed.

[0003] Figure 1 An example system for implementing automatic music audio transcription according to the present disclosure is shown.

[0004] Figure 2 An example transformed spectrogram according to the present disclosure is shown.

[0005] Figure 3 An example deep neural network model for implementing automatic music audio transcription according to the present disclosure is shown.

[0006] Figure 4 An example system for augmenting training data according to the present disclosure is shown.

[0007] Figure 5 An example process for implementing automatic music audio transcription according to the present disclosure is shown.

[0008] Figure 6 An example process for implementing automatic music audio transcription according to the present disclosure is shown.

[0009] Figure 7 An example process for implementing training a deep neural network model configured for automatic music audio transcription according to the present disclosure is shown.

[0010] Figure 8 An example process for implementing automatic music audio transcription according to the present disclosure is shown.

[0011] Fig. 9 An example process for tracking the beats and downbeats of human speech in real time that may be performed in accordance with the present disclosure is shown.

[0012] Fig.10 An example process for tracking the beats and downbeats of human speech in real time that may be performed in accordance with the present disclosure is shown.

[0013] Fig.11 An example computing device is shown that can be used to perform any of the techniques disclosed herein. DETAILED DESCRIPTION

[0014] Musical scores act as a medium between people and music. Musical scores can include notes from a variety of different instruments. Automatic music transcription (AMT) is a music information retrieval (MIR) task that aims to transcribe music audio input into a sequence of notes. Each note in the sequence can be associated with properties such as starting point, pitch, duration, and speed. The output of the AMT system is usually delivered in the format of a MIDI file. If the music audio input includes sounds from a variety of different instruments, the AMT system should identify each instrument present in the music audio input and estimate the associated notes accordingly to a specific channel of the MIDI output. The synthesized audio from the output MIDI can be similar to the original input audio in a musically reasonable way.

[0015] However, AMT is a difficult task. Compared with other music information retrieval (MIR) tasks, AMT has rarely been studied due to the lack of training data sets and model capabilities. Existing AMT systems face two major challenges. First, existing AMT systems lack model scalability. The number of instruments commonly used can be as high as 100. Among them, the notes of conventional instruments such as guitars, violins, and synthesizers are difficult to characterize due to the huge differences in their timbre, expressiveness, and playing techniques. In addition, the human voice, which is usually the most dominant instrument (instrument) (if present), varies in timbre and pitch to convey lyrics and expressions. Some existing AMT systems use a simple (vanilla) transformer architecture, only calculate self-attention on the time axis, and cannot fully capture spectral features from various instruments. Therefore, there is a need for an AMT system with improved model scalability that can transcribe all instruments simultaneously.

[0016] Second, existing AMT systems are generally unable to distinguish between different musical instruments. Many existing AMT systems result in false positive notes for popular pitched instruments such as piano or guitar. For example, notes of a string ensemble may be captured in large quantities by a piano. This may occur if the AMT system does not provide clear timbre-related features and / or if the AMT system is not robust to timbre variations between different instruments. In addition, existing AMT systems primarily focus on the transcription of common musical instruments, ignoring the human voice. However, if present, the human voice is typically one of the most important signal sources in a piece of music. Therefore, there is a need for an AMT system with an improved ability to distinguish between different musical instruments and the human voice.

[0017] An improved AMT system is described herein. The AMT system described herein has improved model scalability. In addition, the AMT system described herein has an improved ability to distinguish between different musical instruments and vocals. Figure 1 An example system 100 according to the present disclosure is illustrated. The system 100 can be used to implement automatic music audio transcription. The system 100 can include a convolutional model 104 and a deep neural network model 106.

[0018] Spectrogram 102 can be created (e.g., generated). Spectrogram 102 can be created based on a music audio clip. The music audio clip can include a song or a part of a song. The music audio clip can include human voice and / or multiple different instrument sounds. Human voice can represent human speech (e.g., singing and / or speaking). It should be noted that human speech is received (including but not limited to receiving, processing, etc.) only when it is authorized by the owner of the human voice to be used. Each instrument sound in multiple different instrument sounds can correspond to a different instrument characterized by a music audio clip. The musical instrument can include one or more of a guitar, piano, drum, bass, cello, violin, piccolo, flute, trumpet, harmonica, xylophone, saxophone, harp and / or any other musical instrument. Spectrogram 102 can include a visual representation of the frequency spectrum of the music audio clip changing over time. Spectrogram 102 can be input into convolution model 104.

[0019] The spectrogram 102 may be passed through a convolutional layer 104 for local feature aggregation. The convolutional model 104 may include a convolutional neural network (CNN). The convolutional model 104 may include a plurality of stacked residual units that utilize average pooling to reduce the dimensionality of the frequency axis. The resulting time-frequency representation may be represented as Where T, F and C represent the time dimension, frequency dimension and channel dimension respectively.

[0020] The convolution model 104 can be configured to transform the spectrogram 102 into a plurality of spectral representations. Each of the plurality of spectral representations can correspond to a particular time step of the music audio segment. Each time step can include a predetermined number of seconds or milliseconds of the music audio segment. For example, the convolution model 104 can transform the spectrogram 102 into a first spectral representation corresponding to a first time step of the music audio segment, a second spectral representation corresponding to a second time step of the music audio segment, a third spectral representation corresponding to a third time step of the music audio segment, and so on.

[0021] Figure 2 An example multiple spectral representation 200 generated by the convolutional model 104 is shown. tThe multiple spectrum representations may be generated by the convolution model 104 based on the spectrogram 102. The multiple spectrum representations carry pitch and timbre information. The multiple spectrum representations may include a first spectrum representation S0, a second spectrum representation S1, ..., S T , …, and S T-1 The first spectral representation S0 may correspond to a first time step of the music audio clip, the second spectral representation S1 may correspond to a second time step of the music audio clip, etc. T and F may represent the length of the time axis and the frequency axis, respectively.

[0022] Return to reference Figure 1 , extracted features 105 may be generated. Features 105 may be extracted from a music audio clip corresponding to spectrogram 102. The extracted features 105 may be generated by extracting audio features from the music audio clip. The extracted features 105 may include, for example, pitch, duration, absolute pitch distance below, absolute pitch distance above, onset position in a measure, pitch in a scale, etc. The extracted features may be mapped to a latent array θ t 0 , where t represents each time step and 0 represents the initialization of the potential array.

[0023] The deep neural network model 106 may include a spectral cross attention sub-model 108, a latent transformer sub-model 110, and a temporal transformer sub-model 112. The deep neural network model 106 may utilize a convolutional neural network (e.g., convolutional layer 104) and a latent array set sequence (e.g., latent array θ t 0 (k)) to generate MIDI output 114. For example, deep neural network model 106 can identify each instrument present in the musical audio input and estimate the associated notes accordingly to a specific channel of MIDI output 114. The synthesized audio from the output MIDI can resemble the original input audio in a musically plausible manner.

[0024] The deep neural network model 106 utilizes cross-attention to extract spectral features into the latent bottleneck of each frame, and adds additional transformers for self-attention along the time axis, resulting in a quadratic complexity overall. Since F is typically large, this complexity reduction is significant, allowing the deep neural network model 106 to process more instruments simultaneously.

[0025] Figure 3 is an example architecture of the deep neural network model 106. Multiple spectral representations S t (For example, S0, S1, ..., S t-1 ) and the potential array θ t 0 105 can be input into the spectral cross attention (SCA) sub-model 108. The SCA sub-model 108 can represent the input spectrum S t Each input spectral representation in is directly operated and projected into Key (K) and Value (V) matrices. Unlike traditional transformers, the SCA sub-model 108 maps the latent array into the Query (Q) matrix and then performs QKV self-attention accordingly. The set of K learnable latent arrays can be initialized with K being the index dimension and D being the channel dimension. Then, the potential array Θ 0 can be repeated T times, and each one is associated with a time step t, denoted as Make Therefore, all potential arrays can come from the same initialization across the timeline. (h represents the hth iteration) plays an important role in carrying the spectrum information from the SCA sub-model 108 in the entire deep neural network model 106. The query-key-value (QKV) attention of the hth iteration of the SCA sub-model 108 can be written as: SCA : This process can be repeated as the deep neural network model 106 is repeated to maintain With input S t Compared with the existing AMT system, the design of SCA sub-model 108 significantly improves the computational scalability of deep neural network model 106. For example, SCA sub-model 108 generates Given that K (the dimension of the underlying array) is typically small (i.e., K<<F), Compared with the spectrum converter (“SpecTNT”) used in the prior art Much easier.

[0026] The output of SCA submodel 108 may be input into potential transformer submodel 110. Potential transformer submodel 110 may include a plurality of potential transformers for refining a potential array. Potential transformer submodel 110 may include a stack of N transformers to refine a potential array. The complexity of performing self-attention on is effective. In the context of AMT, this process means that the interaction between onset, pitch, and instrument is explicitly modeled. To perform multi-track AMT, K latent arrays are initialized and trained, each latent array is trained to handle a specific task. For each instrument, two latent arrays can be arranged to model the onset and frame-by-frame (pitch) activations, respectively. This leads to K=2J, where J is the number of target instruments.

[0027] The output of the latent transformer submodel 110 may be input into a temporal transformer submodel 112. The temporal transformer submodel 112 may include a set of temporal transformers for processing the latent array associated with each particular track at all time steps to learn temporal coherence separately. The temporal transformer may process θ at all time steps. t h , to model the temporal coherence. The temporal transformer can be placed so that t h To make the temporal transformer understand the temporal position of each latent array, a trainable position embedding can be added to each θ during initialization. t 0 Let θ t h (k), k=0,...,K-1 represents θ t h Each potential array in . The time transformer submodel 112 may include K parallel transformers. Each transformer in the K parallel transformers may serve a corresponding potential array input sequence: The time transformer submodel 112 may be repeated M times, resulting in a complexity of

[0028] The above process may be repeated L times to form the entire deep neural network model 106. The weights of the SCA submodel 108 and the potential transformer submodel 110 are not shared between repeated blocks. The output of the deep neural network model 106 may include multiple MIDI data sets or file sets corresponding to any one of the vocals and multi-instrument tracks. For example, the deep neural network model 106 may identify each instrument present in the music audio input and estimate the associated notes accordingly to a specific channel of the MIDI output. The synthesized audio mix from the output MIDI resembles the original input audio in a musically reasonable manner.

[0029] In order to ensure that the deep neural network model 106 has an improved ability to distinguish different musical instruments and vocals, the deep neural network model 106 can be trained using enhanced training data. The enhanced training data can be generated using a random mixing enhancement technique. The purpose of the random mixing enhancement technique is to separate each instrument interlude from the input audio mixture. A combination of different data sets can be used to create more accurate data samples instead of using pseudo labels. The deep neural network model 106 can be trained in a multi-task learning manner, where each subtask models the transcription of an instrument or vocal. This multi-task design together with the random mixing technique allows for more flexible training of a large number of enhanced training samples.

[0030] Figure 4 An example system 400 for enhancing training data using a cross-dataset random mixing (RM) technique is shown. Annotation data for multi-track AMT is labor-intensive. In order to better utilize existing data, two data enhancement techniques can be applied during training. During training, pitch shifting can be performed randomly for all non-percussion instruments. For the cross-dataset RM technique, three different types of data sets can be used. The first type of data set can include a multi-track data set 402. In the multi-track data set 402, each sample contains multiple tracks of instrument-by-instrument audio stems with polyphonic notes. There is no vocal signal in the multi-track data set 402. The second type of data set can include a single-track data set 404. Each sample in the single-track data set 404 contains only a single non-vocal stem with polyphonic notes. The third type of data set can include a vocal mixed data set 406. Each sample in the vocal mixed data set 406 is a complete mixture of music and monophonic notes only for the dominant vocals. A music source separation (MSS) tool can be used to separate each sample into a vocal stem and an accompaniment stem.

[0031] Each training sample can be extracted from a random moment of its original song, where the duration is related to the model input length (e.g., 6 seconds). To transcribe J types of instruments, the corresponding set of instruments can be represented as Then, three treatments can be introduced for three different types of data sets respectively.

[0032] First, for the training example s from the multi-track dataset 402 i , whose instrument template can be expressed as Instructions i Then, for μ i Each instrument in j , there is a p% chance that it will be replaced by μ u ω in j , where i≠u (i.e., different examples). Second, for example s from the single-track dataset 404i , existing instrument template μ u can be randomly selected as its background. If μ u There are i For instruments, the tracks can be divided from μ u Remove. For example, if s i If it is a piano solo, the piano track can be divided from μ u Removed. Presenting solo examples to model training without mixing them with the background may degrade performance. Finally, for example s from the vocal mixture dataset 406 i , it has a q% chance of replacing its background by at least one of two methods: (i) similar to the processing of the single-track dataset 404, the existing μ u (i≠u) can be randomly selected as its background; or (ii) from s v A separate accompaniment track, where i≠v can be randomly selected. For the second method, since the selected accompaniment track does not have underlying true value notes, the instrument output can be masked and only the loss for the vocal output can be counted.

[0033] The following loss function can be used to train the deep neural network model 106 on the augmented dataset:

[0034]

[0035] where l represents the binary cross entropy loss between the ground truth and the prediction, and are the onset and frame activation losses for instrument j, and Mj is a mask determined by the availability of labels for instrument j.

[0036] Figure 5 An example process 500 for implementing automatic music audio transcription is illustrated. Figure 5 2 is depicted as a sequence of operations, but one of ordinary skill in the art will appreciate that various embodiments may add, remove, reorder, or modify the operations depicted.

[0037] At 502, a deep neural network model implementing automatic music transcription may be configured. The deep neural network model may include a spectral cross-attention sub-model, a potential transformer sub-model, and a temporal transformer sub-model. The deep neural network model may utilize cross-attention to extract spectral features into potential bottlenecks for each frame and add additional transformers for self-attention along the time axis, resulting in a significant reduction in complexity overall, thereby allowing the deep neural network model to process more instruments and vocals simultaneously.

[0038] The spectral cross-attention sub-model can be configured to transform the spectral representation (denoted as S t ) is projected onto the latent array at time step t (denoted as θ t h ), h represents the hth iteration. The latent transformer submodel may include a submodel configured to transform the latent array θ t h The latent transformer submodel may also include a set of temporal transformers configured to enable any latent array θ at different time steps to be t h Communicate between pairs.

[0039] At 504, the training data can be enhanced. The training data can be enhanced by randomly mixing multiple types of data sets. Multiple types of data sets can include vocal data sets and instrument data sets. Multiple types of data sets can include the type of multi-track data sets, each sample of which contains multiple tracks of instrument audio sub-tracks with polyphonic notes. Multiple types of data sets can include the type of single-track data sets, each sample of which contains a single non-human voice audio sub-track with polyphonic notes. Multiple types of data sets can include the type of human voice mixed data sets, each sample of which contains a complete mixture of music audio and monophonic notes for leading human voice. The enhanced training data can be generated using random mixing enhancement technology. The purpose of the random mixing enhancement technology is to separate each predefined sub-track source from the input audio mix. A combination of different data sets can be used to create more accurate data samples instead of using pseudo labels. At 506, a deep neural network model can be trained. The deep neural network model can be trained using the enhanced training data. The deep neural network model can be trained to automatically transcribe the music audio including the human voice track and the multi-instrument audio track into multiple MIDI data sets corresponding to each of the human voice audio and the multi-instrument audio. The deep neural network model can be trained in a multi-task learning manner, where each sub-task models the transcription of a musical instrument. This multi-task design together with the random mixing technique allows for more flexible training with a large number of augmented training samples.

[0040] Figure 6 An example process 600 for implementing automatic music audio transcription is illustrated. Figure 6 2 is depicted as a sequence of operations, but one of ordinary skill in the art will appreciate that various embodiments may add, delete, reorder, or modify the depicted operations.

[0041] At 602, K learnable latent arrays θ 0(k) sets can be initialized. K is defined based on the number of target tracks of music. For example, the set of K learnable latent arrays Can be initialized, where K is the index dimension and D is the channel dimension. Each specific track can correspond to two outputs from the deep neural network model. The two outputs can indicate the starting point and frame-by-frame pitch of the specific track. Two of the K learnable potential arrays correspond to the two outputs of the specific track, respectively.

[0042] At 604, θ 0 (k) can be repeated T times and is associated with each time step t, denoted as θ t 0 (k), so that all potential arrays come from the same initialization across the time axis. For example, Θ 0 can be repeated T times and each can be associated with a time step t, and then represented as Make Therefore, all potential arrays can come from the same initialization across the timeline. It can play an important role in carrying spectrum information in the entire deep neural network model.

[0043] The query key value (QKV) attention of the SCA sub-model 108 at the hth iteration can be written as: SCA : This process can be repeated as the deep neural network model 106 is repeated to maintain With input S t The design of the SCA sub-model 108 significantly improves the computational scalability of the deep neural network model 106 compared to existing AMT systems. For example, given that K (the dimension of the potential array) is typically small (i.e., K<<F), the SCA sub-model generates The spectrum converter used in the prior art ("SpecTNT") compared to, Much easier.

[0044] In order for the temporal transformer submodel to understand the temporal position of each potential array, a trainable position embedding may be added to each potential array during initialization. At 606, the trainable position embedding may be added to each potential array during the initiation process with θ t 0 (k) is added to each potential array so that the temporal transformer set can understand the temporal position of each potential array. express Each potential array in . The time transformer submodel may include K parallel transformers. Each transformer in the K parallel transformers may serve a corresponding input sequence of the potential array:

[0045] Figure 7 An example process 700 for implementing training a deep neural network model configured for automatic music audio transcription is illustrated. Figure 7 2 is depicted as a sequence of operations, but one of ordinary skill in the art will appreciate that various embodiments may add, delete, reorder, or modify the depicted operations.

[0046] At 702, a loss function for training a deep neural network model may be formulated. The loss function may be formulated as: where l represents the binary cross entropy loss between the ground truth and the prediction, represents the starting point activation loss for instrument j, represents the frame activation loss for instrument j, and M j Mask representing label availability for instrument j.

[0047] At 704, the training data can be enhanced. The training data can be enhanced by randomly mixing multiple types of data sets. Multiple types of data sets can include vocal data sets and instrument data sets. Multiple types of data sets can include types of multi-track data sets, each sample of which contains multiple tracks of instrument audio sub-tracks with polyphonic notes. Multiple types of data sets can include types of single-track data sets, each sample of which contains a single non-human voice audio sub-track with polyphonic notes. Multiple types of data sets can include types of vocal mixed data sets, each sample of which contains a complete mixture of music audio and monophonic notes for dominant vocals. Enhanced training data can be generated using random mixing enhancement technology. The purpose of random mixing enhancement technology is to separate each predefined sub-track source from the input audio mix. Combinations of different data sets can be used to create more accurate data samples instead of using pseudo labels.

[0048] At 706, the deep neural network model can be trained using the enhanced training data. The deep neural network model can be trained based on a loss function. The deep neural network model can be trained to automatically transcribe music audio including a vocal track and a multi-instrument track into a plurality of MIDI data sets corresponding to each of the vocal track and the multi-instrument track.

[0049] Figure 8 An example process 800 for implementing automatic music audio transcription is illustrated. Figure 8 2 is depicted as a sequence of operations, but one of ordinary skill in the art will appreciate that various embodiments may add, delete, reorder, or modify the depicted operations.

[0050] At 802, training data can be enhanced. Training data can be enhanced by randomly mixing multiple types of data sets. Multiple types of data sets can include vocal data sets and instrument data sets. Multiple types of data sets can include the type of multi-track data sets, each sample of which contains multiple tracks of instrument audio sub-tracks with polyphonic notes. Multiple types of data sets can include the type of single-track data sets, each sample of which contains a single non-human voice audio sub-track with polyphonic notes. Multiple types of data sets can include the type of human voice mixed data sets, each sample of which contains a complete mixture of music audio and monophonic notes for leading human voice. Enhanced training data can be generated using random mixing enhancement techniques. Combinations of different data sets can be used to create more accurate data samples instead of using pseudo labels. Random mixing enhancement techniques enable more flexible training of deep neural networks using a large number of enhanced training samples.

[0051] At 804, a deep neural network model may be trained using the enhanced training data. The deep neural network model may be trained based on a loss function. The deep neural network model may be trained to automatically transcribe music audio including a vocal track and a multi-instrument track into a plurality of MIDI data sets corresponding to each of the vocal track and the multi-instrument track.

[0052] At 806, a music audio clip may be input into a system 100 including a trained deep neural network model. The music audio clip may include vocals and instrument sounds. The music audio clip may include multiple different instrument sounds. At 808, a system including a trained deep neural network model may automatically generate a transcription based on the music audio clip. The transcription may include a MIDI data set, each of which corresponds to one of the vocals and instrument sounds. For example, the trained deep neural network model may identify each instrument present in the music audio input and estimate the associated notes to a specific channel of the MIDI output accordingly. The synthesized audio from the output MIDI may be similar to the original input audio in a musically reasonable manner.

[0053] Described in this article Figure 1 and Figure 3The deep neural network model 106 in is evaluated. The results of the evaluation show that the trained deep neural network model 106 fully solves the model scalability and instrument recognition problems experienced by existing AMT systems. Four public datasets are used for evaluation: for example, a public dataset contains 2100 multi-track MIDI fragments and corresponding synthetic audio. Formal training / validation / test splits are used in the experiment. Another public dataset contains approximately 200 hours of piano solo recordings, in which the aligned note annotations are obtained by a MIDI capture device on the piano. Follow the formal training / validation / test split. Another public dataset contains 360 high-quality guitar recordings and their synchronized note annotations. The first two processes of each style can be used for training, and the last one for testing. Another specific public dataset contains 500 pop songs with note annotations for dominant vocal melodies. Formal training test splits were used.

[0054] The audio waveform may be resampled to a sampling rate of 16KHz. The model input length may be set to 6 seconds. The log-magnitude spectrogram may then be calculated using a Hann window of 2048 samples and a hop size of 320 samples (i.e., 20ms). The convolutional model may contain three residual blocks, each with 128 channels and followed by an average pooling layer with a (1, 2) time-frequency filter.

[0055] For the deep neural network model 106, the following parameters are used: (i) 2J latent arrays are initialized according to different experimental configurations, and each latent array uses a dimension of 128; (ii) L = 3 perceptron TF blocks are stacked; (iii) for each perceptron TF block, 1 spectral cross attention layer, N = 2 latent transformer layers, and M = 2 temporal transformer layers are used. All transformer layers in the transformer layer can have a hidden size of 128, where 8 heads are used for multi-head attention. Finally, the output module that can generate MIDI output is a 2-layer bidirectional GRU with 128 hidden units. All transformer models in the deep neural network model 106 may include a dropout ratio of 0.15. The output dimensions for the starting point and frame activation can be 128 and 129, respectively, where 128 corresponds to MIDI pitch and the additional 1 dimension in the frame activation is used for silence. AdamW is used as a learning optimizer. The initial learning rate and weight decay rate are set to 10, respectively. -3 and 5×10 -3 .

[0056] For the final output, a threshold of 0.25 was used on the onset and frame probability outputs to obtain a binary representation so that the onset and frame-by-frame activations could be merged to generate each note in the piano-roll representation. No further post-processing was applied.

[0057] For data augmentation, all non-percussion channels of training examples have a 100% probability of being pitch-shifted up or down by at most 3 semitones. For random mixing, we use p = 25% and q = 50% for training examples from the multi-track and vocal mixture datasets, respectively.

[0058] Two state-of-the-art models, MT3 and spectnt, are selected as baselines. The “Onset F1” score is evaluated against the proposed model and the baselines. The Onset F1 score indicates the correctness of both pitch and onset timestamps, as an evaluation metric for comparison with previous work. To further evaluate the performance of multi-instrument transcription, the “Multi-instrument Onset F1” score for the Slakh dataset is also evaluated.

[0059] Fig. 9 Table 900 is depicted. Table 900 shows a comparison in terms of Onset F1 between the proposed model and the baseline. The models with (Mix) or (Mix+Vocal) are trained on a mixture of datasets, while the model with (Single) is trained on a single dataset. The proposed model and SpecTNT, which directly model the spectral input using an attention mechanism, show high performance even for the case of training on low resources of a single dataset. On a public vocal dataset, the proposed model significantly outperforms the baseline. Although SpecTNT (Single) performs slightly better than our model on MAE-STRO, the deep neural network model 106 is still more advantageous for practical use due to its better inference efficiency. As shown in Table 900, the deep neural network model for implementing automatic music transcription according to the present disclosure, such as the deep neural network model 106, performs better than the baseline model for each instrument when trained on data augmented using a random mixing technique.

[0060] Fig.10Table 1000 is depicted. Table 1000 presents multi-instrument Onset F1 (instrument weighted average) scores and Onset F1 scores for individual instrument categories on a public multi-track instrument dataset to reveal instrument-by-instrument performance. Compared to MT3, the deep neural network model described in the present disclosure without random mixing enhancement (No-RM) (e.g., deep neural network model 106) performs significantly better (Onset F1 score is more than 100% higher) on less common instruments such as "Pipe". Applying random mixing in training can further improve performance in all cases, indicating that the technique does improve the robustness of the model for distinguishing different instruments. Finally, Table 1000 shows that combining multi-instrument and vocal transcription can improve vocal transcription alone because the combined model is trained using more randomly mixed vocal-accompaniment samples.

[0061] Fig.11 The diagram illustrates a computing device that can be used in various aspects, such as Figure 1 and / or Figure 3 The services, networks, modules and / or devices described in the Figure 1 and / or Figure 3 , any or all of the components can be Fig.11 The system may be implemented by one or more instances of computing device 1100. Fig.11 The computer architecture shown illustrates a conventional server computer, workstation, desktop computer, laptop computer, tablet computer, network appliance, PDA, e-reader, digital cellular telephone, or other computing node and can be used to perform any aspects of the computer described herein, such as to implement the methods described herein.

[0062] The computing device 1100 may include a baseboard or "motherboard," which is a printed circuit board to which multiple components or devices may be connected via a system bus or other electrical communication paths. One or more central processing units (CPUs) 1104 may operate in conjunction with a chipset 1106. The CPU(s) 1104 may be a standard programmable processor that performs the arithmetic and logic operations required for the operation of the computing device 1100.

[0063] The CPU(s) 1104 can perform the necessary operations by switching from one discrete physical state to the next by manipulating switching elements that distinguish between and change these states. Switching elements can typically include electronic circuits that maintain one of two binary states, such as flip-flops, and electronic circuits that provide output states based on logical combinations of the states of one or more other switching elements (such as logic gates). These basic switching elements can be combined to create more complex logic circuits, including registers, adder-subtractors, arithmetic logic units, floating point units, etc.

[0064] CPU(s) 1104 may be augmented or replaced with other processing units such as GPU(s) 1105. GPU(s) 1105 may include processing units specialized for, but not necessarily limited to, highly parallel computations such as graphics and other visualization-related processing.

[0065] The chipset 1106 may provide an interface between the CPU(s) 1104 and the rest of the components and devices on the substrate. The chipset 1106 may provide an interface to a random access memory (RAM) 1108 used as main memory in the computing device 1100. The chipset 1106 may also provide an interface to a computer-readable storage medium such as a read-only memory (ROM) 1120 or a non-volatile RAM (NVRAM) (not shown) for storing basic routines that may help boot the computing device 1100 and transfer information between various components and devices. The ROM 1120 or NVRAM may also store other software components required for the operation of the computing device 1100 in accordance with various aspects described herein.

[0066] The computing device 1100 may operate in a networked environment using logical connections to remote computing nodes and computer systems via a local area network (LAN). The chipset 1106 may include functionality for providing network connectivity via a network interface controller (NIC) 1122, such as a Gigabit Ethernet adapter. The NIC 1122 may be capable of connecting the computing device 1100 to other computing nodes via the network 1116. It should be understood that multiple NICs 1122 may be present in the computing device 1100, connecting the computing device to other types of networks and remote computer systems.

[0067] The computing device 1100 may be connected to a mass storage device 1128 that provides non-volatile storage for the computer. The mass storage device 1128 may store system programs, application programs, other program modules, and data that have been described in more detail herein. The mass storage device 1128 may be connected to the computing device 1100 by means of a storage controller 1124 connected to the chipset 1106. The mass storage device 1128 may include one or more physical storage units. The mass storage device 1128 may include a management component 1110. The storage controller 1124 may interface with a physical storage unit by means of a serial attached SCSI (SAS) interface, a serial advanced technology attachment (SATA) interface, a fiber channel (FC) interface, or other types of interfaces for physically connecting and transmitting data between a computer and a physical storage unit.

[0068] The computing device 1100 can store data on the mass storage device 1128 by transforming the physical state of the physical storage unit to reflect the stored information. The specific transformation of the physical state may be related to various factors and different implementations of the present description. Examples of such factors may include, but are not limited to, the technology used to implement the physical storage unit and whether the mass storage device 1128 is characterized as primary or secondary storage, etc.

[0069] For example, the computing device 1100 can store information to the mass storage device 1128 by issuing instructions through the storage controller 1124 to change the magnetic properties of specific locations in the disk drive unit, the reflective or refractive properties of specific locations in the optical storage unit, or the electrical properties of specific capacitors, transistors, or other discrete components in the solid-state storage unit. Other transformations of physical media are also possible without departing from the scope and spirit of the present specification, and the foregoing examples are provided only for the convenience of this description. The computing device 1100 can also read information from the mass storage device 1128 by detecting the physical state or characteristics of one or more specific locations in the physical storage unit.

[0070] In addition to the mass storage device 1128 described above, the computing device 1100 may access other computer-readable storage media to store and retrieve information, such as program modules, data structures, or other data. It should be understood by those skilled in the art that computer-readable storage media may be any available media that provides storage of non-transitory data and can be accessed by the computing device 1100.

[0071] By way of example and not limitation, computer-readable storage media may include volatile and nonvolatile, temporary computer-readable storage media and non-transitory computer-readable storage media, and removable and non-removable media implemented in any method or technology. Computer-readable storage media include, but are not limited to, RAM, ROM, erasable programmable ROM ("EPROM"), electrically erasable programmable ROM ("EEPROM"), flash memory or other solid-state memory technology, compact disk ROM ("CD-ROM"), digital versatile disk ("DVD"), high-definition DVD ("HD-DVD"), BLU-RAY or other optical storage devices, cassettes, magnetic tapes, magnetic disk storage devices, other magnetic storage devices, or any other medium that can be used to store the desired information in a non-transitory manner.

[0072] Mass storage devices (such as Fig.11 The depicted mass storage device 1128) can store an operating system for controlling the operation of the computing device 1100. The operating system can include a version of the LINUX operating system. The operating system can include a version of the WINDOWSSERVER operating system from Microsoft Corporation. According to a further aspect, the operating system can include a version of the UNIX operating system. Various mobile phone operating systems, such as IOS and ANDROID, can also be used. It should be understood that other operating systems can also be used. The mass storage device 1128 can store other systems or applications and data used by the computing device 1100.

[0073] The mass storage device 1128 or other computer-readable storage medium may also be encoded with computer-executable instructions that, when loaded into the computing device 1100, transform the computing device from a general-purpose computing system into a special-purpose computer capable of implementing the aspects described herein. As described above, these computer-executable instructions transform the computing device 1100 by specifying how the CPU(s) 1104 transition between states. The computing device 1100 may access a computer-readable storage medium storing computer-executable instructions that, when executed by the computing device 1100, may perform the methods described herein.

[0074] Computing devices (such as Fig.11The computing device 1100 shown in the figure may also include an input / output controller 1132, which is used to receive and process input from a number of input devices, such as a keyboard, a mouse, a touch pad, a touch screen, an electronic stylus, or other types of input devices. Similarly, the input / output controller 1132 can provide output to a display, such as a computer monitor, a flat panel display, a digital projector, a printer, a plotter, or other types of output devices. It should be understood that the computing device 1100 may not include Fig.11 All components shown may include Fig.11 Other components not explicitly shown may be used in conjunction with Fig.11 The architecture shown is a completely different architecture.

[0075] As described herein, a computing device may be a physical computing device such as Fig.11 The computing device 1100 of the present invention may also include a virtual machine host process and one or more virtual machine instances. The computer executable instructions may be indirectly executed by the physical hardware of the computing device by interpreting and / or executing instructions stored and executed in the context of the virtual machine.

[0076] It should be understood that the methods and systems are not limited to specific methods, specific components or specific implementations. It should also be understood that the terminology used herein is only used for the purpose of describing specific embodiments and is not intended to be limiting.

[0077] Unless the context clearly indicates otherwise, as used in the specification and the appended claims, the singular forms "a", "an", and "the" include plural referents. Ranges may be expressed herein as from "about" one particular value, and / or to "about" another particular value. When such a range is expressed, another embodiment includes from a particular value and / or to another particular value. Similarly, when a value is expressed as an approximation by using the antecedent "about", it is understood that the particular value forms another embodiment. It is also understood that the endpoints of each range are significant both relative to the other endpoint and independently of the other endpoint.

[0078] "Optional" or "optionally" means that the subsequently described event or circumstance may or may not occur, and that the description includes instances where said event or circumstance occurs and instances where it does not.

[0079] Throughout the description and claims of this specification, the word "comprise" and variations of the word, such as "comprising" and "comprises", mean "including but not limited to", and are not intended to exclude, for example, other components, integers or steps. "Exemplary" means "as an example of..." and is not intended to convey an indication of a preferred or ideal embodiment. "Such as" is not used in a limiting sense, but for explanatory purposes.

[0080] Components that can be used to perform the described methods and systems are described. When combinations, subsets, interactions, groups, etc. of these components are described, it is understood that, although specific references to each of the various individual and collective combinations and arrangements of these components may not be explicitly described, each is specifically contemplated and described herein for all methods and systems. This applies to all aspects of the application, including but not limited to the operations in the described methods. Therefore, if there are various additional operations that can be performed, it is understood that each of these additional operations can be performed using any specific embodiment or combination of embodiments of the described methods.

[0081] The method and system of the present invention may be understood more readily by reference to the following detailed description of the preferred embodiments and examples included therein and to the accompanying drawings and their description.

[0082] Those skilled in the art will appreciate that the methods and systems may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Additionally, the methods and systems may take the form of a computer program product on a computer-readable storage medium having computer-readable program instructions (e.g., computer software) embodied in the storage medium. More specifically, the methods and systems may take the form of web-implemented computer software. Any suitable computer-readable storage medium may be used, including a hard disk, a CD-ROM, an optical storage device, or a magnetic storage device.

[0083] Embodiments of the method and system are described below with reference to the block diagrams and flow charts of the method, system, device and computer program product. It should be understood that each frame in the block diagram and flow chart and the combination of frames in the block diagram and flow chart can be implemented by computer program instructions respectively. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer or other programmable data processing device to produce a machine, so that the instructions executed on the computer or other programmable data processing device create a device for implementing the functions specified in one or more flow chart frames.

[0084] These computer program instructions may also be stored in a computer-readable memory, which may instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture, which includes computer-readable instructions for implementing the functions specified in one or more flowchart blocks. The computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operating steps are performed on the computer or other programmable device, thereby producing a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flowchart blocks.

[0085] The various features and processes described above can be used independently of each other or can be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. Additionally, certain method or process blocks can be omitted in some implementations. The methods and processes described herein are also not limited to any specific sequence, and the blocks or states associated therewith can be performed in other appropriate sequences. For example, the described blocks or states can be performed in an order different from that specifically described, or multiple blocks or states can be combined in a single block or state. Example blocks or states can be performed serially, in parallel, or in some other way. Blocks or states can be added to or removed from the described example embodiments. The example systems and components described herein can be configured differently from those described. For example, compared with the described exemplary embodiments, elements can be added to, removed from, or rearranged from the described exemplary embodiments.

[0086] It should also be understood that various items are illustrated as being stored in memory or on storage when used, and that these items or portions thereof may be transferred between memory and other storage devices for the purpose of memory management and data integrity. Alternatively, in other embodiments, some or all of the software modules and / or systems may be executed in memory on another device and communicate with the illustrated computing system via inter-computer communication. In addition, in some embodiments, some or all of the systems and / or modules may be implemented or provided in other ways, such as at least partially implemented or provided in firmware and / or hardware, including but not limited to one or more application specific integrated circuits (“ASICs”), standard integrated circuits, controllers (e.g., by executing appropriate instructions and including microcontrollers and / or embedded controllers), field programmable gate arrays (“FPGAs”), complex programmable logic devices (“CPLDs”), etc. Some or all of the modules, systems, and data structures may also be stored (e.g., as software instructions or structured data) on a computer-readable medium (e.g., a hard disk, memory, a network, or a portable media product) to be read by an appropriate device or via an appropriate connection. Systems, modules, and data structures may also be transmitted as generated data signals (e.g., as part of a carrier wave or other analog or digital propagation signal) on various computer-readable transmission media (including wireless and wire / cable-based media), and may take various forms (e.g., as part of a single or multiplexed analog signal, or as multiple discrete digital packets or frames). In other embodiments, such computer program products may also take other forms. Therefore, the present invention may be implemented using other computer system configurations.

[0087] Although the method and system have been described in conjunction with preferred embodiments and specific examples, it is not intended to limit the scope to the particular embodiments set forth, as the embodiments herein are intended in all respects to be illustrative rather than restrictive.

[0088] Unless otherwise expressly stated, it is not intended that any method set forth herein be construed as requiring that its operations be performed in a particular order. Thus, in the absence of a method claim actually reciting the order in which its operations are to be followed or otherwise specifically stating in the claims or specification that the operations are to be limited to a particular order, no order is intended to be inferred in any respect. This applies to any possible non-express basis for interpretation, including: logical issues regarding the arrangement of steps or operational flow; ordinary meaning derived from grammatical organization or punctuation; and the number or type of embodiments described in the specification.

[0089] It is obvious to those skilled in the art that various modifications and variations may be made without departing from the scope or spirit of the present disclosure. Other embodiments will be apparent to those skilled in the art in view of the specification and practice described herein. The specification and example drawings are to be considered as exemplary only, with the true scope and spirit being indicated by the appended claims.

Claims

1. A method for implementing automatic music audio transcription, comprising: A deep neural network model is configured to implement the automatic music transcription, wherein the deep neural network model includes a spectral cross-attention sub-model, and the spectral cross-attention sub-model is configured to project the spectral representation of each time step t to a potential array set at the time step t, wherein the spectral representation is denoted as St and the potential array is denoted as θ t h , h represents the hth iteration, wherein the deep neural network model includes a plurality of potential transformers, and the plurality of potential transformers are configured to transform the potential array θ t h The set performs self-attention, and wherein the deep neural network model further includes a set of time transformers configured to enable at different time steps, at any potential array θ t h To communicate between pairs; The training data is enhanced by randomly mixing multiple types of datasets, wherein the multiple types of datasets include a vocal dataset and a musical instrument dataset; as well as The deep neural network model is trained using the enhanced training data, wherein the deep neural network model is trained to automatically transcribe music audio including a vocal track and a multi-instrument track into a plurality of MIDI data sets corresponding to each of the vocal track and the multi-instrument track.

2. The method according to claim 1, wherein configuring the deep neural network model comprises: Initiate K learnable latent arrays θ 0 (k) sets, where K is defined based on the number of target tracks of music; and θ 0 (k) is repeated T times and is associated with each time step t, thus denoted as θ t 0 (k), so that all potential arrays come from the same initialization across the time axis.

3. The method according to claim 2, further comprising: During the initiation process, a trainable position embedding is added to each latent array so that the set of temporal transformers can understand the temporal position of each latent array.

4. The method of claim 2, wherein each of the K learnable potential arrays corresponds to a specific audio track, wherein the set of time transformers includes K time transformers, wherein each of the K time transformers serves a corresponding input sequence of the potential array along the time axis, and wherein the corresponding input sequence of the potential array along the time axis includes θ0 h (k),θ1 h (k),…,θ T-1 h (k).

5. The method of claim 4, wherein each specific audio track corresponds to two outputs from the deep neural network model, wherein the two outputs indicate a starting point pitch and a frame-by-frame pitch of the specific track, and wherein two of the K learnable potential arrays correspond to the two outputs of the specific audio track, respectively.

6. The method according to claim 1, wherein the plurality of types of data sets include: A type of a multi-track data set, each sample of which contains multiple tracks of instrument audio tracks having polyphonic notes; The types of mono-track datasets, each sample of which contains a single non-vocal audio track with polyphonic notes; and the types of vocal mixed datasets, each sample of which contains a complete mix of musical audio with monophonic notes for the dominant vocal.

7. The method according to claim 1, further comprising: The loss function for training the deep neural network model is formulated, wherein the loss function is formulated as: where l represents the binary cross entropy loss between the ground truth and the prediction, represents the starting point activation loss for instrument j, denotes the frame activation loss for instrument j, and Mj denotes the mask determined by the availability of labels for instrument j.

8. A method according to claim 1, wherein the output of the spectral cross-attention sub-model is input into the multiple latent transformers for refining each set of latent arrays at each time step, and wherein the output of the multiple latent transformers is input into the set of temporal transformers for processing the latent arrays associated with each specific audio track at all time steps to learn temporal coherence.

9. The method according to claim 1, further comprising: Inputting a music audio clip into a system including the trained deep neural network model, wherein the music audio clip includes human voice and instrumental sounds; as well as Automatically generating a transcription based on the music audio clip through the trained deep neural network model, wherein the transcription includes a MIDI data set, each of the MIDI data sets corresponding to one of the human voice and the instrument sound.

10. A system for extracting melody, comprising: at least one processor; as well as at least one memory, the at least one memory being communicatively coupled to the at least one processor and comprising computer-readable instructions that, when executed by the at least one processor, cause the at least one processor to perform operations comprising: A deep neural network model is configured to implement the automatic music transcription, wherein the deep neural network model includes a spectral cross-attention sub-model, and the spectral cross-attention sub-model is configured to project the spectral representation of each time step t to a potential array set at the time step t, wherein the spectral representation is denoted as St and the potential array is denoted as θ t h , h represents the hth iteration, wherein the deep neural network model includes a plurality of potential transformers, and the plurality of potential transformers are configured to transform the potential array θ t h The set performs self-attention, and wherein the deep neural network model further includes a set of time transformers configured to enable at different time steps, at any potential array θ t h To communicate between pairs; enhancing the training data by randomly mixing a plurality of types of datasets, wherein the plurality of types of datasets include a vocal dataset and a musical instrument dataset; and The deep neural network model is trained using the enhanced training data, wherein the deep neural network model is trained to automatically transcribe music audio including a vocal track and a multi-instrument track into a plurality of MIDI data sets corresponding to each of the vocal track and the multi-instrument track.

11. The system of claim 10, wherein configuring the deep neural network model comprises: Initiate K learnable latent arrays θ 0 (k) sets, where K is defined based on the number of target tracks of music; and θ 0 (k) is repeated T times and is associated with each time step t, thus denoted as θ t 0 (k), so that all potential arrays come from the same initialization across the time axis.

12. The system of claim 11, the operations further comprising: During the initiation process, a trainable position embedding is added to each latent array so that the set of temporal transformers can understand the temporal position of each latent array.

13. The system of claim 11, wherein each of the K learnable latent arrays corresponds to a particular audio track, wherein the set of time transformers comprises K time transformers, wherein each of the K time transformers serves a corresponding input sequence of the latent array along the time axis, and wherein the corresponding input sequence of the latent array along the time axis comprises θ0 h (k),θ1 h (k),…,θ T-1 h (k).

14. The system of claim 13, wherein each specific audio track corresponds to two outputs from the deep neural network model, wherein the two outputs indicate a starting point pitch and a frame-by-frame pitch of the specific track, and wherein two of the K learnable latent arrays correspond to the two outputs of the specific audio track, respectively.

15. The system of claim 10, wherein the output of the spectral cross-attention sub-model is input into the plurality of latent transformers for refining each set of latent arrays at each time step, and wherein the output of the plurality of latent transformers is input into the set of temporal transformers for processing the latent arrays associated with each particular audio track at all time steps to learn temporal coherence.

16. A non-transitory computer-readable storage medium storing computer-readable instructions, which when executed by a processor causes the processor to perform operations comprising: A deep neural network model is configured to implement the automatic music transcription, wherein the deep neural network model includes a spectral cross-attention sub-model, and the spectral cross-attention sub-model is configured to project the spectral representation of each time step t to a potential array set at the time step t, wherein the spectral representation is denoted as St and the potential array is denoted as θ t h , h represents the hth iteration, wherein the deep neural network model includes a plurality of potential transformers, and the plurality of potential transformers are configured to transform the potential array θ t h The set performs self-attention, and wherein the deep neural network model further includes a set of time transformers configured to enable at different time steps, at any potential array θ t h To communicate between pairs; The training data is enhanced by randomly mixing multiple types of datasets, wherein the multiple types of datasets include a vocal dataset and a musical instrument dataset; as well as The deep neural network model is trained using the enhanced training data, wherein the deep neural network model is trained to automatically transcribe music audio including a vocal track and a multi-instrument track into a plurality of MIDI data sets corresponding to each of the vocal track and the multi-instrument track.

17. The non-transitory computer-readable storage medium of claim 16, wherein configuring the deep neural network model comprises: Initiate K learnable latent arrays θ 0 (k) sets, where K is defined based on the number of target tracks of music; and θ 0 (k) is repeated T times and is associated with each time step t, thus denoted as θ t 0 (k), so that all potential arrays come from the same initialization across the time axis.

18. The non-transitory computer-readable storage medium of claim 17, wherein each of the K learnable potential arrays corresponds to a specific audio track, wherein the set of time transformers includes K time transformers, wherein each of the K time transformers serves a corresponding input sequence of the potential array along the time axis, and wherein the corresponding input sequence of the potential array along the time axis includes θ0 h (k),θ1 h (k),…,θ T-1 h (k).

19. The non-transitory computer-readable storage medium of claim 18, wherein each particular audio track corresponds to two outputs from the deep neural network model, wherein the two outputs indicate a starting point pitch and a frame-by-frame pitch of the particular track, and wherein two of the K learnable potential arrays correspond to the two outputs of the particular audio track, respectively.

20. A non-transitory computer-readable storage medium according to claim 16, wherein the output of the spectral cross-attention sub-model is input into the multiple potential transformers for refining each set of potential arrays at each time step, and wherein the output of the multiple potential transformers is input into the set of temporal transformers for processing the potential arrays associated with each specific audio track at all time steps to learn temporal coherence.

Citation Information

Cited By

  • Multi-track MIDI audio transcription method, MIDI audio processing method and system

    CN122658339A