Method and system for generating theme condition music based on dynamic gating
By introducing a dynamic gating mechanism, the problem of abrupt switching of thematic influence in existing music generation models is solved, and the natural integration of thematic and non-thematic segments is achieved, resulting in musical works with more coherent structure and more natural thematic development, thus improving the quality and controllability of music generation.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHAANXI NORMAL UNIV
- Filing Date
- 2026-01-26
- Publication Date
- 2026-05-01
AI Technical Summary
In existing music generation models based on theme conditions, the use of hard XOR gating results in abrupt switching of theme influence and an inability to achieve natural integration of theme development and non-theme segments. This leads to auditory breaks or splicing at the boundaries between the theme and non-theme regions, disrupting the natural flow and gradual development of the music.
A learnable soft gating mechanism is introduced, which dynamically adjusts the influence of the topic on the generation process, enabling the model to dynamically adjust the influence weight of the topic according to the musical context. A dynamic gating topic-conditional Transformer model is adopted, which includes a topic encoder and a dynamic gating decoder. A multilayer perceptron is used to calculate the fusion weights to achieve a natural transition between topic and non-topic segments.
It generates musical works with more natural thematic development and more coherent structure, provides interpretability for model decisions, and does not require a significant increase in computational overhead. It has good practicality and scalability, and promotes the development of music generation towards a higher level of musical understanding.
Smart Images

Figure CN121963673A_ABST
Abstract
Description
A Thematic Conditional Music Generation Method and System Based on Dynamic Gating Technical Field
[0001] This invention belongs to the field of intelligent music generation, specifically relating to a method and system for generating theme-conditional music based on dynamic gating. Background Technology
[0002] The core of musical composition lies in the creative development and variation of musical themes. Composers typically begin with a core musical motif or theme, using techniques such as repetition, transformation, and development to construct a structurally complete and expressive musical work. This theme-based creative paradigm is a crucial source of music's inherent logic and emotional tension. However, with the widespread application of artificial intelligence technology, especially deep learning, in music generation, enabling machine models to understand and simulate this theme-based creative process has become a key challenge.
[0003] In recent years, Transformer models based on attention mechanisms have achieved significant success in symbolic music generation. Their powerful long-range dependency modeling capabilities make it possible to generate music with good local coherence. Currently, most mainstream conditional generation methods are "cue-based generation," where user-provided musical fragments serve as starting prefixes, which the model then continues autonomously. While this method can generate stylistically coherent music, it inherently cannot guarantee that the model will consciously reproduce, echo, or develop the initially given musical material in subsequent development. The generated music often lacks a clear structure built around a core theme on a macro level, easily falling into the predicament of "lack of direction" or the diminishing influence of the theme during the generation process, which is far removed from the organized and hierarchical creative model of human composers around a theme. A patent with publication number CN115762449A provides a method and system for automatically generating conditional music theme melodies based on Transformer. This method addresses the lack of harmony and regularity in music generation. It converts a MIDI music dataset into a theme music dataset, uses a Transformer encoder and decoder to encode and decode music theme fragments, and combines multi-head self-attention and cross-attention mechanisms to generate melodies with musical theme styles, improving music quality and diversity and solving the gradient vanishing problem in recurrent neural networks when processing long sequences. However, this type of model has a key technical bottleneck in its access mechanism for theme memory: the weight allocation of self-attention and cross-attention paths typically employs a static, binary, hard gating strategy. Specifically, a typical implementation is based on a preset theme mask. If the current decoding position is marked as a "thematic region," the output of the self-attention path is completely suppressed, and only features carrying thematic information from the cross-attention path are used. Conversely, in the "non-thematic region," the cross-attention path is completely blocked, and only the self-attention path is relied upon to maintain the internal coherence of the sequence. The appearance and disappearance of the theme manifest as instantaneous changes in weights, resulting in a perceptible auditory "break" or "splitting" at the boundary between thematic and non-thematic regions in the generated music. This disrupts the natural flow and gradual development of the music. Therefore, a thematic conditional generation method based on dynamic gating is proposed. Summary of the Invention
[0004] To address the problems in existing theme-based music generation models, where the use of hard XOR gating leads to abrupt switching of theme influence and an inability to achieve natural integration of theme development and non-theme segments, this invention provides a theme-based music generation method and system based on dynamic gating. By introducing learnable soft gating, the model can dynamically adjust the influence of the theme on the generation process according to the musical context, thereby generating musical works with more natural theme presentation and more coherent structure.
[0005] To achieve the above objectives, in a first aspect, the present invention provides a method for generating theme-conditional music based on dynamic gating, comprising the following steps: using a basic theme-conditional Transformer model as a baseline model, and on the basis of the baseline model, replacing the cascaded attention modules in the decoder layer with self-attention modules and cross-attention modules to obtain a dynamically gated theme-conditional Transformer model; the dynamically gated theme-conditional Transformer model includes a theme encoder and a dynamically gated decoder, wherein the theme encoder uses a Transformer encoder to encode the theme; the theme encoder includes multiple encoder layers, each encoder layer including a self-attention mechanism and a feedforward neural network, and employing a bidirectional attention mechanism to process the theme. The sequence of topics is used to obtain a high-dimensional representation of the topic. The dynamic gated decoder consists of multiple improved decoder layers, each containing a self-attention module, a cross-attention module, and a dynamic gated fusion module. In each parallel attention layer of the dynamic gated decoder, a gating network is added. The gating network takes the self-attention output, cross-attention output, and current music context information as input, and dynamically calculates the attention fusion weights at each time step through two fully connected layers and a sigmoid activation function. Using the optimized dynamic gated topic-conditional Transformer model, the encoder receives topic fragments as input and extracts deep representations of topic fragments through 6 layers of bidirectional self-attention. The decoder receives the entire music sequence and generates music step by step through 6 decoder layers.
[0006] Further optimization of the dynamically gated topic-conditional Transformer model includes: training the model using a public music dataset and tuning hyperparameters on a validation set; evaluating the model's performance on a test set using topic consistency, topic uncontrollability, and topic variability, and comparing it with a baseline model; visualizing the dynamic gating weights to verify the dynamic changes in topic influence across different music segments; and further adjusting the structure and parameters of the dynamic gating network based on the evaluation results, iteratively optimizing the model to obtain the optimized dynamically gated topic-conditional Transformer model. The model uses a negative log-likelihood loss function during training and optimizes parameters through backpropagation and the Adam optimizer, with a learning rate of 2×10⁻⁶. -4 The batch size is 8.
[0007] Furthermore, the self-attention module in the decoder layer adopts a unidirectional attention mechanism, focusing only on the generated sequence portion; the cross-attention module in the decoder layer interacts with the decoder state and the topic representation output by the encoder, calculates the correlation between the decoder state and the topic representation, and extracts topic information related to the current generation.
[0008] Furthermore, the input to the dynamic gated decoder includes word embedding representation, position encoding, and topic region mask. The word embedding representation converts the music event token into a 256-dimensional vector. The position encoding uses sine function encoding to provide temporal information in the music sequence. The topic region mask is a binary vector used to indicate whether the current generated position is in the topic region, where a value of 1 indicates that it is in the topic region and a value of 0 indicates that it is in a non-topic region.
[0009] Furthermore, the calculation process of the dynamic gating fusion module includes: concatenating the self-attention output, cross-attention output, and current context representation into a vector; calculating the fusion weights through a multilayer perceptron, which contains two linear layers and an activation function, with the output being a scalar between 0 and 1; and using the calculated fusion weights to perform a weighted summation of the self-attention output and cross-attention output to obtain the fused representation.
[0010] Furthermore, the specific structure of the multilayer perceptron in the dynamic gating fusion module is as follows: the first linear layer maps the concatenated input vector to a space with the same hidden dimension; the Tanh activation function is used to perform a nonlinear transformation on the output of the first linear layer; the second linear layer maps the nonlinearly transformed features to a scalar output; and the Sigmoid function is used to constrain the output between 0 and 1 to generate dynamic gating weights.
[0011] Furthermore, the current context representation is obtained by calculating a context encoder, which is implemented using a linear layer. The input is the concatenation of the hidden state of the previous time step and the current input, and the output is the current context representation.
[0012] Secondly, this invention provides a theme-conditional music generation system based on dynamic gating, comprising a model building module and a music generation module. The model building module uses a basic theme-conditional Transformer model as a baseline model. Based on the baseline model, it replaces the cascaded attention module in the decoder layer with a self-attention module and a cross-attention module to obtain a dynamically gated theme-conditional Transformer model. The dynamically gated theme-conditional Transformer model includes a theme encoder and a dynamically gated decoder. The theme encoder uses a Transformer encoder to encode the theme. The theme encoder includes multiple encoder layers, each including a self-attention mechanism and a feedforward neural network, employing a bidirectional attention mechanism. The algorithm processes the topic sequence to obtain a high-dimensional representation of the topic. The dynamic gated decoder consists of multiple improved decoder layers, each containing a self-attention module, a cross-attention module, and a dynamic gated fusion module. In each parallel attention layer of the dynamic gated decoder, a gating network is added. The gating network takes the self-attention output, cross-attention output, and current music context information as input, and dynamically calculates the attention fusion weights at each time step through two fully connected layers and a sigmoid activation function. The music generation module uses an optimized dynamic gated topic-conditional Transformer model. The encoder receives topic fragments as input and extracts deep representations of the topic fragments through six bidirectional self-attention layers. The decoder receives the entire music sequence and generates music step by step through six decoder layers.
[0013] Thirdly, the present invention also provides a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads part or all of the computer executable program from the memory and executes it, and the processor can implement the above-mentioned theme conditional music generation method based on dynamic gating when executing part or all of the computer executable program.
[0014] A computer-readable storage medium is also provided, in which a computer program is stored. When the computer program is executed by a processor, it can implement the above-described method for generating theme-conditional music based on dynamic gating.
[0015] Compared with existing technologies, this invention has at least the following advantages: By replacing the rigid XOR gating in the basic theme-conditional Transformer model with learnable dynamic soft gating, this invention enables the model to dynamically adjust the influence weights of themes according to the musical context, achieving a natural transition between theme materials and free creation, and generating works with more reasonable theme development and more coherent musical structure. Visualizing the dynamic gating weights provides interpretability for the model's decisions, helping to understand how the model utilizes theme information during the generation process. The dynamic gating mechanism can be flexibly integrated into existing Transformer decoder architectures without significantly increasing computational overhead, exhibiting good practicality and scalability. This method provides a more refined means of conditional control for controllable music generation, promoting the development of music generation towards a higher level of musical understanding. Attached Figure Description
[0016] Figure 1 is a flowchart of the present invention.
[0017] Figure 2 is a schematic diagram of the audio processing part of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Referring to Figure 1, the present invention provides a method for generating theme-conditional music based on dynamic gating, comprising the following steps: using the POP909 dataset, which contains multi-track MIDI files of 909 popular piano pieces. Each song contains three tracks: melody, accompaniment, and bridge. The average length of each song is approximately 95 measures, and it uses a 4 / 4 time signature. First, the POP909 dataset is preprocessed, quantizing the note start time and duration to make them multiples of 1 / 4 time, and filtering out songs with internal atonality, ultimately resulting in 713 songs for training and testing.
[0020] Furthermore, a theme fragment is extracted from each song as a conditional input. Each song is divided into two non-overlapping measures, retaining only the measure containing the melody. Using contrastive learning, each song is first cut into two non-overlapping measures along bar lines, and these two measures are converted into embedding vectors. The contrastive learning employs three data augmentation strategies: uniformly shifting the pitch of the scale by the same fixed interval, randomly varying the duration of the last note, and splitting the note—randomly selecting a note and dividing it into two notes with the same pitch and half the original duration. Then, the DBSCAN clustering algorithm is used to cluster the fragments in each song, i.e., clustering the embedding vectors corresponding to all fragments. The distance from all fragments to the cluster center must be less than a certain threshold, resulting in multiple clusters. The earliest fragment in the largest cluster is selected as the theme fragment of the song, which contains an average of 121.9 tokens.
[0021] Furthermore, a theme-conditional music generation model is constructed. This model adopts an encoder-decoder architecture, where the encoder receives theme fragments as input, extracts the deep representation of the theme fragments through six layers of bidirectional self-attention, and converts the MIDI-format theme fragments into embedding vectors; simultaneously, positional encoding is added to preserve temporal order information. The encoder is a standard Transformer encoder stacked with six layers, each layer containing a multi-head self-attention network and a feedforward network. Bidirectional self-attention allows each token to simultaneously attend to all tokens before and after it, thereby capturing the relationship between the melodic direction, rhythmic organization, and local motifs within the theme fragment; the decoder receives the entire music sequence as input and generates music step by step through six decoder layers. The design of the decoder layers is the core of this invention. Each decoder layer contains a self-attention module, a cross-attention module, and a dynamic gating fusion module. The self-attention module focuses on the internal structure of the generated music sequence, using a unidirectional attention mechanism to ensure the causality of the generation process; the cross-attention module focuses on the deep representation of the encoder output, using a bidirectional attention mechanism to extract theme information relevant to the current generation; the dynamic gating fusion module dynamically fuses the self-attention output and the cross-attention output according to the current music context.
[0022] As a further optimization, the decoder input consists of three parts: word embedding representation, position encoding, and topic region mask. The word embedding representation converts the music event token into a 256-dimensional vector; the position encoding uses a sine function to represent the temporal information in the token sequence; the topic region mask is a binary vector used to indicate whether the current generation position is in a topic region, where a value of 1 indicates that it is in a topic region and a value of 0 indicates that it is in a non-topic region.
[0023] Furthermore, during training, a negative log-likelihood loss function is used, and the model parameters are optimized through backpropagation and the Adam optimizer, with the learning rate set to 2×10⁻⁶. -4 The batch size is 8. During the generation phase, an autoregressive method is used to gradually generate the music sequence, generating one music event token each time, until the preset length of 64 bars is reached.
[0024] This method automatically identifies theme fragments from the original music through a theme extraction algorithm. It achieves an organic combination of theme conditions and free creation through a dynamic gating mechanism, enabling the model to follow the theme more closely in the theme area and have greater creative freedom in the non-theme area, thereby generating musical works with reasonable structure and natural theme development.
[0025] Referring to Figure 2, the classifier model training specifically includes: passing the input sequence (including topic conditions and generated music events) sequentially through a word embedding layer, a position encoding layer, and a topic mask label to form the decoder input representation. The decoder is composed of multiple improved decoder layers stacked together. The core of each decoder layer is a dynamic gating fusion module. In each decoder layer, the input representation is fed into the self-attention path and the cross-attention path in parallel: (1) The self-attention path adopts a unidirectional multi-head attention mechanism, calculates the internal dependency relationship based on the currently generated music sequence, and outputs a self-attention feature vector; (2) The cross-attention path adopts a bidirectional multi-head attention mechanism, interacts with the topic representation output by the encoder, and extracts feature information related to the topic; further, the output feature vectors of the above two paths are concatenated along the feature dimension and input into the dynamic gating network. Referring to Figure 2, the dynamic gating network is implemented using a multilayer perceptron, specifically including: 1) The first linear layer receives a feature vector from the previous processing stage as input. This feature vector is composed of three parts: a 256-dimensional feature vector output from the self-attention module, a 256-dimensional feature vector output from the cross-attention module, and a 256-dimensional feature vector generated by the context encoder. These three parts are concatenated into a 256-dimensional joint vector. The first linear layer takes this 256-dimensional vector as input and maps it to a 256-dimensional hidden space through a fully connected transformation matrix. The purpose of this step is to project the high-dimensional fused features into a space consistent with the core hidden dimension of the model, providing a unified dimensional representation for subsequent nonlinear feature extraction. This process completes the first dimensional transformation from multi-source feature fusion representation to standardized hidden representation.
[0026] 2) The Tanh activation function layer receives the 256-dimensional feature vector output from the first linear layer as input. A hyperbolic tangent function is independently applied to each element of this vector for a non-linear transformation. The Tanh function maps the input values to the range [-1, 1]. This non-linear activation process enables the network to learn and express complex interactions between features, enhancing the model's ability to perceive subtle differences in musical context. After the Tanh transformation, the distribution of feature values is normalized to a symmetric interval, which helps improve gradient flow and prevents gradient explosion or vanishing problems that may occur in deep networks, providing stable and normalized feature input for the next stage of weight generation.
[0027] 3) The second linear layer receives the 256-dimensional feature vector processed by the Tanh activation function as input. This second linear layer uses another fully connected transformation matrix to map the 256-dimensional feature vector into a single scalar value. The weight matrix of this linear transformation has a dimension of 1×256, compressing the multi-dimensional features into a single-dimensional numerical representation. This scalar value carries the model's comprehensive evaluation information on the relative importance of the self-attention path and the cross-attention path within the current time step, and can be regarded as the "raw score" or logical value of the gating weights. This step completes the dimensionality compression from a multi-dimensional feature representation to a single importance evaluation value.
[0028] 4) The Sigmoid activation function layer receives a single scalar value from the output of the second linear layer as input. The Sigmoid function is applied to this scalar, mapping it to the interval [0, 1]. This transformation converts the unbounded logistic value into a gating weight with a definite probabilistic interpretation. The output dynamic gating weight is a scalar between 0 and 1, generated independently at each decoding time step. This weight serves as a coefficient in subsequent fusion steps, dynamically determining the mixing ratio of self-attention output and cross-attention output in the final feature representation: when the weight is close to 1, the model relies more on topic-conditional memory; when the weight is close to 0, the model relies more on generated content memory; intermediate values represent a smooth transition and blending of the two.
[0029] This invention breaks away from the simple prompt-and-continue paradigm, constructing a model capable of using a given theme as core material and learning to explicitly reproduce and appropriately vary it multiple times in generated music. The model requires a dual-memory mechanism: it must both remember already generated content to maintain coherence and continuously remember and reference the given conditional theme. By introducing a dynamic gating attention and theme alignment mechanism, the model can dynamically and selectively draw inspiration from a theme library during the generation process, thereby generating polyphonic musical works with clearer structures and more logical thematic development, propelling controllable music generation towards a higher level of musical understanding and expression.
[0030] Based on the above method, this invention provides a dynamically gated topic-conditional music generation system, including a model building module and a music generation module. The model building module uses a basic topic-conditional Transformer model as a baseline model. Based on the baseline model, it replaces the cascaded attention module in the decoder layer with a self-attention module and a cross-attention module to obtain a dynamically gated topic-conditional Transformer model. The dynamically gated topic-conditional Transformer model includes a topic encoder and a dynamically gated decoder. The topic encoder uses a Transformer encoder to encode the topic. The topic encoder includes multiple encoder layers, each including a self-attention mechanism and a feedforward neural network, employing a bidirectional attention machine. The system processes the topic sequence to obtain a high-dimensional representation of the topic. The dynamic gated decoder consists of multiple improved decoder layers, each containing a self-attention module, a cross-attention module, and a dynamic gated fusion module. In each parallel attention layer of the dynamic gated decoder, a gating network is added. The gating network takes the self-attention output, cross-attention output, and current music context information as input, and dynamically calculates the attention fusion weights at each time step through two fully connected layers and a sigmoid activation function. The music generation module uses an optimized dynamic gated topic-conditional Transformer model. The encoder receives topic fragments as input and extracts deep representations of the topic fragments through six bidirectional self-attention layers. The decoder receives the entire music sequence and generates music step by step through six decoder layers.
[0031] On the other hand, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can implement the theme conditional music generation method based on dynamic gating described in the present invention.
[0032] The present invention can also provide a computer device, including a processor and a memory, wherein the memory is used to store a computer executable program, the processor reads the computer executable program from the memory and executes it, and the processor can implement the theme conditional music generation method based on dynamic gating described in the present invention when executing the computer executable program.
[0033] The computer device may be a laptop, tablet, desktop computer, or workstation.
[0034] The processor can be a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), or an off-the-shelf programmable gate array (FPGA).
[0035] The memory described in this invention can be an internal storage unit of a laptop, tablet, desktop computer, or workstation, such as memory or hard disk; or it can be an external storage unit, such as a portable hard disk or flash memory card.
[0036] Computer-readable storage media can include computer storage media and communication media. Computer storage media includes volatile and non-volatile, removable and non-removable media implemented using any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media can include: read-only memory (ROM), random access memory (RAM), solid-state drives (SSDs), or optical discs, etc. Random access memory can include resistive random access memory (ReRAM) and dynamic random access memory (DRAM).
[0037] The above content is only for illustrating the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solution based on the technical concept proposed in this invention shall fall within the scope of protection of the claims of this invention.
Claims
1. A method for generating theme-conditional music based on dynamic gating, characterized in that, Includes the following steps: Using the basic topic-conditional Transformer model as the baseline model, the cascaded attention modules in the decoder layer are replaced with self-attention and cross-attention modules to obtain the dynamically gated topic-conditional Transformer model. This dynamically gated model includes a topic encoder and a dynamically gated decoder. The topic encoder uses a Transformer encoder to encode topics. The topic encoder contains multiple encoder layers, each including a self-attention mechanism and a feedforward neural network, employing a bidirectional attention mechanism to process the topic sequence and obtain a high-dimensional representation of the topics. The dynamically gated decoder consists of multiple improved decoder layers, each containing a self-attention module, a cross-attention module, and a dynamically gated fusion module. In each parallel attention layer of the dynamic gated decoder, a gating network is added. The gating network takes the self-attention output, cross-attention output, and current music context information as input, and dynamically calculates the attention fusion weights at each time step through two fully connected layers and a sigmoid activation function. Using the optimized dynamic gated topic-conditional Transformer model, the encoder receives topic fragments as input and extracts deep representations of topic fragments through 6 bidirectional self-attention layers. The decoder receives the entire music sequence and generates music step by step through 6 decoder layers.
2. The method for generating theme-based conditional music based on dynamic gating according to claim 1, characterized in that, The optimization of the dynamically gated topic-conditional Transformer model includes: training the model using a public music dataset and tuning its hyperparameters on a validation set; evaluating its performance on a test set using topic consistency, topic uncontrollability, and topic variability, and comparing it with a baseline model; visualizing the dynamic gating weights to verify the dynamic changes in topic influence across different music segments; further adjusting the structure and parameters of the dynamic gating network based on the evaluation results, iteratively optimizing the model to obtain the optimized dynamically gated topic-conditional Transformer model; and using a negative log-likelihood loss function during training, with parameter optimization achieved through backpropagation and the Adam optimizer.
3. The method for generating theme-conditional music based on dynamic gating according to claim 1, characterized in that, The self-attention module in the decoder layer adopts a unidirectional attention mechanism, focusing only on the generated sequence portion; the cross-attention module in the decoder layer interacts with the decoder state and the topic representation output by the encoder, calculates the correlation between the decoder state and the topic representation, and extracts topic information related to the current generation.
4. The method for generating theme-based conditional music based on dynamic gating according to claim 1, characterized in that, The input to the dynamic gated decoder includes word embedding representation, position encoding, and topic region mask. The word embedding representation converts the music event token into a 256-dimensional vector. The position encoding uses sine function encoding to provide temporal information in the music sequence. The topic region mask is a binary vector used to indicate whether the current generated position is in the topic region, where a value of 1 indicates that it is in the topic region and a value of 0 indicates that it is in a non-topic region.
5. The method for generating theme-conditional music based on dynamic gating according to claim 1, characterized in that, The calculation process of the dynamic gating fusion module includes: concatenating the self-attention output, cross-attention output, and current context representation into a vector; calculating the fusion weights through a multilayer perceptron, which contains two linear layers and an activation function, with the output being a scalar between 0 and 1; and using the calculated fusion weights to perform a weighted summation of the self-attention output and cross-attention output to obtain the fused representation.
6. The method for generating theme-conditional music based on dynamic gating according to claim 5, characterized in that, The specific structure of the multilayer perceptron in the dynamic gating fusion module is as follows: the first linear layer maps the concatenated input vector to a space with the same hidden dimension; the Tanh activation function is used to perform a nonlinear transformation on the output of the first linear layer. The second linear layer maps the nonlinearly transformed features to a scalar output; the output is constrained between 0 and 1 by the Sigmoid function, generating dynamic gating weights.
7. The method for generating theme-conditional music based on dynamic gating according to claim 5, characterized in that, The current context representation is calculated by a context encoder, which is implemented using a linear layer. The input is the concatenation of the hidden state of the previous time step and the current input, and the output is the current context representation.
8. A theme-based conditional music generation system based on dynamic gating, characterized in that, The system includes a model building module and a music generation module. The model building module uses a basic topic-conditional Transformer model as a baseline model. Based on the baseline model, it replaces the cascaded attention module in the decoder layer with a self-attention module and a cross-attention module to obtain a dynamically gated topic-conditional Transformer model. The dynamically gated topic-conditional Transformer model includes a topic encoder and a dynamically gated decoder. The topic encoder uses a Transformer encoder to encode topics. The topic encoder contains multiple encoder layers, each of which includes a self-attention mechanism and a feedforward neural network. A bidirectional attention mechanism is used to process the topic sequence to obtain a high-dimensional representation of the topic. The dynamically gated decoder consists of multiple improved decoder layers, each of which includes a self-attention module, a cross-attention module, and a dynamically gated fusion module. In each parallel attention layer of the dynamic gated decoder, a gating network is added. The gating network takes the self-attention output, cross-attention output, and current music context information as input, and dynamically calculates the attention fusion weights at each time step through two fully connected layers and a sigmoid activation function. The music generation module uses an optimized dynamic gated topic-conditional Transformer model. The encoder receives topic fragments as input and extracts deep representations of topic fragments through 6 bidirectional self-attention layers. The decoder receives the entire music sequence and generates music step by step through 6 decoder layers.
9. A computer device, characterized in that, It includes a processor and a memory, the memory being used to store a computer-executable program, the processor reading part or all of the computer-executable program from the memory and executing it, and the processor executing part or all of the computer-executable program being able to implement the theme conditional music generation method based on dynamic gating as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, A computer-readable storage medium stores a computer program that, when executed by a processor, enables the generation of theme-conditional music based on dynamic gating as described in any one of claims 1-7.
Citation Information
Patent Citations
Conditional music theme melody automatic generation method and system based on Transform
CN115762449A