Artificial intelligence-based music melody automatic generation system

Abstract

The application relates to the technical field of artificial intelligence-based music melody automatic generation systems, and particularly discloses an artificial intelligence-based music melody automatic generation system. The system comprises an intention analysis and feature mapping module, which is used for converting a user intention into a structured semantic feature; an emotion evolution state machine module, which is used for dynamically generating a time-series emotion state vector according to the semantic feature; a hierarchical controllable generation network module, which is used for generating a note sequence with specific styles, coherent emotions and diversity based on the semantic feature and the emotion state vector; and a post-processing and music theory constraint module, which is used for performing compliance correction and optimization on the note sequence. The application can generate a melody with coherent emotion evolution logic, high matching of a user intention, various styles and compliance with music theory.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, specifically relating to an automatic music melody generation system based on artificial intelligence. Background Technology

[0002] Artificial intelligence (AI) technology has been widely applied in content creation. Its core lies in learning the inherent patterns in data through algorithmic models to automatically create text, images, and even audio content. Music, as an important art form, sees its automatic melody generation as a key branch of AI applications in the creative industries.

[0003] Artificial intelligence-based automatic music melody generation systems aim to use machine learning models to automatically create melodic sequences that conform to music theory and auditory aesthetics based on given input conditions. The basic principle of such systems is typically to train models to learn note sequence patterns and structural features from a large number of existing musical works.

[0004] Existing technologies mainly rely on statistical models or deep learning models to generate note sequences, but they generally suffer from the following problems: the melodies generated by the models lack coherence and evolutionary logic in emotional expression, making it difficult to form complete musical phrases with a clear emotional direction; the system's understanding of the user's high-level intent is insufficient, resulting in a low degree of matching between the generated melodies and the user's expected style, emotion, or scene requirements; in addition, due to limitations in the distribution of training data or model architecture, the generated results are prone to style homogenization, lacking diversity and creativity.

[0005] These problems severely limit the effectiveness of AI music generation systems in practical creative assistance and personalized content production, resulting in generated works that are often mechanical and rigid, failing to meet professional or personalized creative needs. Therefore, there is an urgent need for an intelligent music melody generation solution that can generate emotionally coherent, accurately match user intent, and offer diverse styles. Summary of the Invention

[0006] The purpose of this invention is to provide an artificial intelligence-based automatic music melody generation system to solve the problems in the existing technology, such as the lack of coherence and evolutionary logic in the emotional expression of generated melodies, insufficient understanding of the user's high-level intentions leading to low matching degree, and homogeneity and lack of diversity in the style of generated results.

[0007] This invention provides an artificial intelligence-based automatic music melody generation system, the system comprising: The intent parsing and feature mapping module is used to receive and parse the creative intent information in the form of text, tags or audio input by the user, and transform it into a structured high-level semantic feature vector. The emotional evolution state machine module is used to initialize the emotional state vector based on the high-level semantic feature vector, and dynamically and temporally update the emotional state vector during the melody generation process according to the preset emotional evolution dynamic model, so as to drive the emotional direction of the melody. The hierarchical controllable generative network module is used to receive the real-time emotional state vector output by the emotional evolution state machine module, and based on this vector and the high-level semantic feature vector, it generates a note sequence with specific style, emotional coherence and diversity through a multi-layer neural network architecture including a style decoupling encoder, an emotional conditionalization generator and a diversity injection unit. The post-processing and music theory constraint module is used to apply music theory-based hard rule constraints and harmonic optimization processing to the original note sequence output by the hierarchical controllable generation network module, ensuring the compliance of the generated melody with music theory and the harmony of sound.

[0008] Furthermore, the parsing process of the intent parsing and feature mapping module is as follows: when the user input is text, a pre-trained language model is used to extract the semantic embedding of the text, and a fully connected mapping network is used to project it onto a high-dimensional semantic feature vector aligned with the music feature space; When the user input is audio, the Mel-spectral features of the audio are extracted and encoded into the high-level semantic feature vector by a convolutional neural network encoder; When the user inputs style, mood, or scene tags, the discrete tags are mapped to continuous semantic feature vectors through a learnable tag embedding table.

[0009] Furthermore, the emotion evolution state machine module includes an emotion state vector register and an emotion state transition function; The emotional state vector register is used to store the emotional state vector at the current moment. This emotional state vector is a multi-dimensional real number vector, and its different dimensions represent different aspects of the emotion. The emotional state transition function is a parameterized recurrent neural network whose inputs are the emotional state vector of the previous time step, the target emotional guidance vector derived from the high-level semantic feature vector at the current time step, and the learnable emotional evolution inertia coefficient. The emotional state transition function calculates and outputs the updated emotional state vector at the current moment, thereby achieving a smooth and logical transition and evolution of emotional state over time.

[0010] Furthermore, the style decoupling encoder in the hierarchical controllable generative network module is used to decouple the hidden layer representation of the basic generative model pre-trained on a large and diverse music dataset. This style decoupling encoder learns to separate musical representations into a style-independent content latent space and a style-dependent style latent space through adversarial training. The style decoupled encoder receives the intermediate representation of the base model as input and outputs the decoupled content encoding and style encoding.

[0011] Furthermore, the sentiment conditionalization generator in the hierarchical controllable generative network module is based on a sequence-to-sequence model with an attention mechanism. The decoder part of the sentiment conditionalization generator is conditionalized by the content encoding, style encoding, and the sentiment state vector provided by the sentiment evolution state machine module at the current time step when generating notes in its initial hidden state. Specifically, through a conditional fusion gating mechanism, the emotional state vector and content-style joint encoding are weighted and fused to form the conditional context vector of the decoder, thereby ensuring that each generated note is directly affected by the current emotional state.

[0012] Furthermore, the diversity injection unit in the hierarchical controllable generation network module is integrated into the sampling process of the emotion conditionalization generator; After the decoder outputs the probability distribution of the next note, the diversity injection unit does not directly select the note with the highest probability, but introduces a controllable temperature parameter and an exclusion penalty term based on historically generated segments. Temperature parameters are used to adjust the smoothness of the probability distribution, while the exclusion penalty term is used to reduce the selection probability of frequently occurring notes or interval combinations in recently generated note sequences, thereby effectively breaking the inherent generation inertia of the model and increasing the unexpectedness and creativity of the melody.

[0013] Furthermore, the specific processing performed by the post-processing and music theory constraint module includes pitch range limiting, rhythmic time value quantization, prohibition of illegal interval continuity, and harmony optimization; Harmonic optimization is achieved through a lightweight harmonic analysis submodule. This submodule performs real-time harmonic analysis on the generated melody fragments, identifies their implicit chord progressions, and automatically makes minimal adjustments to the pitch of individual notes to correct harmonic errors when it detects that the chord progressions do not conform to the preset harmonic rules, such as consecutive fifths or octaves in parallel progression. At the same time, it preserves the outline and emotional expression of the original melody to the greatest extent possible.

[0014] Furthermore, the system operates in an iterative loop manner, and its workflow is as follows: The intent parsing and feature mapping module parses user input, outputs high-level semantic feature vectors, and sends them to the emotion evolution state machine module for initialization. Next, the emotion evolution state machine module outputs a series of emotion state vectors in time step sequence according to the emotion evolution dynamics model; Meanwhile, the hierarchical controllable generative network module receives these temporal emotional state vectors and high-level semantic feature vectors, and gradually generates the corresponding musical note sequences. Finally, the post-processing and music theory constraint module performs compliance corrections and optimizations on the complete note sequence, outputting the final musical melody data.

[0015] Furthermore, the emotional evolution dynamics model supports a variety of preset emotional evolution templates, including linear incremental type, wave oscillation type, conflict resolution type and stable maintenance type; Users can select an appropriate emotional evolution template through intention input or the system can automatically match it based on high-level semantic features. This emotional evolution template affects the calculation logic of the target emotional guidance vector and the emotional evolution inertia coefficient in the emotional state transition function in the form of parameters, thereby achieving macro-control over the overall emotional development of the melody.

[0016] Furthermore, the training process of the system is divided into three stages: In the first stage, the basic generative model and the style decoupled encoder were pre-trained using a large-scale unlabeled music dataset; In the second stage, music datasets with style and emotion labels were used to jointly fine-tune the emotion conditionalization generator, the emotion evolution state machine module, and the intent parsing and feature mapping module. In the third stage, a reinforcement learning strategy based on music theory rules and diversity evaluation indicators is introduced to further optimize the parameters of the diversity injection unit and some parameters of the generator in order to improve the music quality and creativity of the generated results.

[0017] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention transforms vague high-level user creative intentions into precise and computable structured semantic features by constructing an intent parsing and feature mapping module, providing clear directional guidance for the subsequent generation process and fundamentally improving the matching accuracy between system output and user expectations.

[0018] 2. This invention innovatively introduces an emotional evolution state machine module, which models emotions as internal states that evolve dynamically over time, rather than static labels. This emotional evolution state machine module simulates the logical change process of emotions through emotional state transition functions, so that the generated melody can present an emotional curve with a beginning, development, transition and conclusion, and a development path. This completely solves the problem of mechanical and fragmented emotional expression in the prior art, and realizes the coherence and artistic expressiveness of the melody in the emotional dimension.

[0019] 3. This invention employs a hierarchical controllable generative network module. Through the collaborative design of style decoupling, emotional conditional generation, and diversity injection, it achieves precise decoupling and independent control over musical style, emotional expression, and creative diversity. The style decoupling encoder ensures the purity of style control, the emotional conditional generator guarantees the deep integration of emotional expression, and the diversity injection unit effectively breaks the model's generation pattern. The combination of these three elements enables the system to generate melodies that highly conform to specific style and emotional requirements while avoiding homogenization, significantly improving the diversity and novelty of the generated works.

[0020] 4. The post-processing and music theory constraint module in this invention serves as a quality assurance link in the generation process. By applying hard constraints from music theory and harmonic optimization, it ensures that regardless of how the front-end model is generated, the final output conforms to basic music composition norms and has a harmonious auditory effect. This combines the creativity of artificial intelligence with rigorous music theory, enhancing the practicality and professionalism of the generated results. The entire system architecture is clearly structured, with each module having a defined function and working collaboratively to form an intelligent automatic music melody generation solution capable of understanding intent, interpreting emotions, and creating diverse and music theory-compliant melodies. Attached Figure Description

[0021] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the emotion evolution state machine module in this invention; Figure 3 This is a logical flowchart of the hierarchical controllable generation network module in this invention; Figure 4 This is a schematic diagram of the multi-level interaction relationship and data flow between the intent parsing and feature mapping module and the hierarchical controllable generation network module in this invention; Figure 5 This is a flowchart illustrating the main stages of the post-processing and music theory constraint module in this invention for optimizing the original note sequence. Detailed Implementation

[0022] Example 1: The overall technical architecture of the AI-based automatic music melody generation system described in this invention is shown in the attached figure. Figure 1 As shown in the figure. The system consists of four core functional units: an intent parsing and feature mapping module, an emotion evolution state machine module, a hierarchical controllable generation network module, and a post-processing and music theory constraint module. These modules collaborate through strictly defined data interfaces and temporal logic, forming a closed-loop, controllable, and highly artistically expressive intelligent melody generation pipeline. The following will combine the attached... Figure 1 To be continued Figure 5The document provides a detailed explanation of the specific implementation mechanism, internal substructure, data flow logic, and parameter configuration of each module.

[0023] First, the intent parsing and feature mapping module, as the system's input perception front-end, is responsible for receiving creative intent information provided by users in the form of text, audio, or discrete tags, and transforming it into a high-dimensional, continuous semantic feature vector aligned with the music semantic space. The internal structure of this feature mapping module contains three parallel parsing paths, each corresponding to one of the three input modalities.

[0024] When the user inputs natural language text, the system calls a language model pre-trained on a large corpus (such as a Chinese semantic encoder based on the Transformer architecture) to perform deep semantic parsing on the natural language text and extract its context-aware semantic embedding vector. This semantic embedding vector is then fed into a mapping sub-network consisting of three fully connected neural networks. The weights of this mapping sub-network are obtained through joint fine-tuning during the second stage of training in the system, and its output dimension is fixed at 512 dimensions to ensure consistency with the feature space dimension of subsequent modules.

[0025] When the user inputs a reference audio, the system first preprocesses the reference audio, including unifying the sampling rate to 44100 Hz, merging the audio channels into a mono channel, and adding a windowed frame (frame length 2048 points, frame shift 512 points). Subsequently, the Mel spectrogram of each frame is extracted to form a time-frequency two-dimensional feature matrix. This time-frequency two-dimensional feature matrix is ​​input into a 5-layer convolutional neural network encoder, with each layer containing 64 to 256 convolutional kernels. ReLU activation and batch normalization are used, and finally, a global average pooling layer compresses it into a 512-dimensional high-level semantic feature vector. When the user inputs discrete tags of style, emotion, or scene (such as "melancholy," "jazz," "early morning," etc.), the system maintains a learnable tag embedding table. This tag embedding table is randomly assigned during the initialization phase and continuously optimized through backpropagation during training. Each tag corresponds to a unique 512-dimensional embedding vector in the table, and query operations are completed directly through the tag index.

[0026] Regardless of the input modality, the intent parsing and feature mapping modules ultimately output a high-level semantic feature vector of the same dimension, denoted as... .for The set of real numbers. This high-level semantic feature vector not only carries the user's explicit creative intent, but also implicitly contains multi-dimensional semantic information such as stylistic tendencies, emotional tone, and scene atmosphere, providing global guidance signals for subsequent emotion modeling and melody generation. Please refer to the appendix. Figure 4 The figure clearly shows the high-level semantic feature vector. How to simultaneously feed the emotion evolution state machine module and the hierarchical controllable generative network module to form a dual-drive data flow.

[0027] Secondly, the emotion evolution state machine module is the core mechanism for achieving emotional coherence and dynamic evolution in this system. (See attached...) Figure 2 As shown, this emotion evolution state machine module consists of two parts: an emotion state vector register and an emotion state transition function. The emotion state vector register is used to store the current time step. Emotional state vector Each dimension of this emotional state vector corresponds to a basic emotional dimension (such as pleasure, arousal, tension, anticipation, etc.), and its numerical range is constrained within the interval [-1, 1], implemented using the tanh activation function. Initial time... Emotional state vector at time It is not randomly initialized, but rather determined by high-level semantic feature vectors. It is obtained through mapping by two layers of perceptron, that is , , For learnable weight matrix, , This is a bias term. This design ensures that the initial emotional state is highly aligned with the user's intent.

[0028] The emotional state transition function is a gated recurrent unit (GRU) network, whose input includes three items: the emotional state at the previous time step. Current moment's target emotion guidance vector And the inertia coefficient of emotional evolution Target sentiment guidance vector It is not constant, but dynamically generated according to a preset emotional evolution template. The system has four built-in standard templates: linear increasing type, wave oscillation type, conflict resolution type, and stable maintenance type.

[0029] For example, under a linearly increasing template, , From The unit vector of emotional development direction decoded in the middle. The evolution rate is 0.02 (typically 0.02); under the wave oscillation template, ,amplitude angular frequency With phase All by The inertia coefficient of emotional evolution was predicted using a small MLP network. The response speed of the control system to target guidance. The larger the number, the more likely the emotional state is to maintain its original trajectory, and the smoother the changes. The smaller the value, the more easily the system is influenced by goal guidance and can quickly change course. This emotional evolution inertia coefficient is also determined by… The predictions ensure that different creative intentions correspond to different emotional dynamics.

[0030] The update formula for the emotion state transition function is as follows: ; For time step The emotional state vector. Input concatenated vector. The dimension is 256, the number of GRU hidden units is 128, and its internal reset gate and update gate mechanism automatically adjusts the fusion ratio between the historical state and the current target.

[0031] Through a recursive process, the emotional state vector A continuous, smooth trajectory with a clear development logic is formed on the timeline, rather than abrupt transitions. This trajectory serves as the real-time emotional context, which is passed step-by-step to the hierarchical controllable generation network module, driving the emotional expression of each note. The entire emotional evolution process is entirely guided by the user's initial intention S, and the macro-emotional context is generated in a controllable manner through a dynamic model.

[0032] The hierarchical controllable generation network module is the backbone engine for melody generation, and its logical flow is shown in the attached figure. Figure 3 As shown, this hierarchical controllable generative network module employs a three-stage hierarchical architecture: a style decoupling encoder, an emotion conditional generator, and a diversity injection unit. First, the style decoupling encoder does not directly participate in real-time generation but is built during the system's first-stage pre-training. It operates on the intermediate layer activations of a base sequence generation model (e.g., MusicTransformer) pre-trained on a million-level MIDI dataset.

[0033] Specifically, the output of the 6th layer decoder of this basic model is taken as the original music representation. The style decoupled encoder consists of two parallel, fully connected sub-networks: the content encoder and the content encoder. With style encoder The system introduces a discriminator through an adversarial training strategy. Its goal is to distinguish Whether it comes from the actual style distribution; at the same time, Forced to be style-independent, meaning that given samples with the same melodic content but different styles... The output should remain consistent. After training is complete, and It is frozen and used only as a feature extractor during the generation phase.

[0034] In actual operation, the system queries the style coding library (built during the training phase and containing typical codes for hundreds of styles) based on the user's intent to obtain the corresponding style coding. Meanwhile, content encoding The content is initially generated by a lightweight content generator based on user intent. This content generator is a two-layer LSTM network that outputs a 16-time-step content sketch.

[0035] The sentiment conditional generator is the core generation unit, essentially an improved Transformer decoder. This Transformer decoder has eight layers, each containing a multi-head self-attention mechanism and a feedforward network. The key innovation lies in its initial hidden state conditionalization mechanism. At each time step... The decoder generates the first Before each note, a conditional context vector needs to be constructed. This vector is composed of three parts: content encoding. (Linear projection to 128 dimensions), style coding (After linear projection to 128 dimensions) and current emotional state (Linearly projected to 128 dimensions). The fusion process is achieved through a gating mechanism: ; For the sigmoid function, For element-wise multiplication, , , , All are learnable weight matrices. This gating mechanism dynamically adjusts the contribution weights of the three components, for example, when... When changes are drastic, the emotional gating weight is automatically increased, making the generation more sensitive to emotional fluctuations. Used as the initial key-value pairs in the first layer of the decoder, thereby deeply integrating high-level semantics, style preferences, and real-time sentiment information into the generation process.

[0036] Finally, the diversity injection unit acts on the probability distribution of the decoder output. The decoder at time step... Output a probability distribution with a dimension of 128. This corresponds to 128 possible note events (including compound symbols such as pitch, duration, and rest). Traditional methods directly sample... This can easily lead to repetition and monotony. This system introduces two control mechanisms: one is temperature sampling, which modifies the original score... Divide by temperature parameter ( ,Depend on (Prediction), to obtain the smoothed distribution Second, the distribution of exclusionary punishment. Define the set of notes that have appeared within the recent history window (8 time steps in length). ,right China belongs to Apply a penalty factor to the element ( ),Right now ,like ,otherwise The final sampling is based on This mechanism effectively suppresses the excessive repetition of high-frequency notes, encourages the exploration of novel combinations with low probability but consistent with the context, and significantly enhances the unexpectedness and creativity of the melody.

[0037] Fourth, the post-processing and music theory constraint module, acting as a quality assurance terminal, performs compliance corrections on the original note sequences output by the layered, controllable generation network. (See attached...) Figure 5 As shown, the post-processing and music theory constraint module executes a four-stage processing flow.

[0038] The first stage is pitch range limiting: based on the user-specified instrument type (such as piano, violin) or default settings, all note pitches are limited to the effective range (e.g., A0 to C8 for piano, corresponding to MIDI notes 21 to 108), and notes outside the range are clamped to the boundary values.

[0039] The second stage is rhythmic timing quantization: continuously generated timing values ​​(in seconds) are mapped to the nearest legal musical timing unit (such as 16th note, 8th note, etc.). The precision of the quantization grid is determined by the time signature (such as 4 / 4, 3 / 4) and tempo (BPM) set by the user.

[0040] The third stage is to prevent the continuous detection of illegal intervals: The system maintains a list of illegal interval pairs (such as an augmented second followed by a diminished seventh, consecutive perfect fifths in parallel, etc.), traverses all adjacent interval pairs in the sequence, and triggers correction if a match is found.

[0041] The correction strategy is as follows: prioritize adjusting the pitch of the next note, selecting the alternative pitch that is closest to the original pitch and does not constitute a new illegal interval; if no feasible solution is found, backtrack and adjust the previous note. The fourth stage is for harmony optimization: the lightweight harmony analysis submodule slides through a 4-bar window to analyze the melody, using a hidden Markov model to infer the most likely chord progression sequence; if a violation of classical harmony rules is detected (such as consecutive fifths or octaves in parallel progression, unresolved leading tones, etc.), a local optimization algorithm is activated.

[0042] This local optimization algorithm uses minimizing pitch variation as the optimization objective and constructs an integer linear programming problem with constraints including: The corrected note must belong to the current chord or its extension; The variation in the melody outline is less than a semitone; No new illegal intervals are introduced.

[0043] The solver employs a greedy approximation algorithm, completing corrections within 10 milliseconds to ensure real-time performance. Through this four-stage processing, the original sequence is transformed into a high-quality melody that retains the original concept while conforming to music theory standards.

[0044] The entire system operates in an iterative loop. During the initialization phase, user input is processed by the intent parsing and feature mapping module, outputting S; S is simultaneously fed into the emotion evolution state machine module to initialize E0, and then into the hierarchical controllable generative network module to obtain... and Subsequently, the system proceeded according to time steps. Sequential execution: Emotional evolution state machine module calculation The hierarchical controllable generative network module is based on , , Generate the first A note; when ( When specified by the user or determined by the generator's self-termination mechanism, the complete note sequence is sent to the post-processing module for optimization, and finally outputs a melody file in standard MIDI or MusicXML format.

[0045] The system training is divided into three stages.

[0046] Phase 1: Using a dataset containing 2 million unlabeled MIDI files, the basic generative model and style-decoupled encoder were self-supervised pre-trained with a loss function of negative log-likelihood.

[0047] Phase 2: Using a labeled dataset of 500,000 songs with style and emotion labels, end-to-end fine-tuning was performed on the intent parsing module, the emotion evolution state machine, and the emotion conditionalization generator. The loss function was the cross-entropy of the generated notes plus the emotion state prediction error.

[0048] Phase 3: Introducing reinforcement learning and defining the reward function. , , , These are the weighting coefficients. Calculated inversely proportional to the number of rule violations in the post-processing module. Based on the n-gram diversity index, By calculating the acoustic characteristics of the generated melody and Trajectory correlation is obtained; the PPO algorithm is used to optimize the diversity injection unit. , Parameters and partial attention weights of the generator.

[0049] The three-stage training ensures that the system retains its ability to generalize big data while accurately aligning with user intent, emotional logic, and musical norms.

[0050] In summary, this embodiment achieves precise capture of high-level semantics through the intent parsing and feature mapping module, constructs a dynamic and coherent emotional development trajectory through the emotion evolution state machine module, realizes decoupled control and diversity stimulation of style, content, and emotion through the hierarchical controllable generation network module, and ensures professional compliance of the output through the post-processing and music theory constraint module. These four modules are tightly coupled, with clear data flow and rigorous control logic, together forming a complete artificial intelligence system capable of generating emotionally rich, stylistically distinctive, structurally sound, and creatively rich musical melodies.

[0051] Example 2: Building upon Example 1, this example customizes and extends the system for a specific application scenario—film score creation assistance—focusing on enhancing the cross-modal alignment capabilities and real-time interactivity of scene-emotion-melody. In this scenario, user input is typically a video clip or script description, and the system needs to generate a background melody highly synchronized with the emotional state of the visuals within a very short time.

[0052] First, the intent parsing and feature mapping module adds a video multimodal parsing subunit. When the input is video, the system extracts keyframes at a frequency of 2 frames per second and performs visual semantic analysis on each frame: image features are extracted using ResNet-50, and then inter-frame dynamic information is aggregated through a spatiotemporal attention network to form a visual semantic vector. Simultaneously, the audio track of the video (if any) is extracted, and Mel spectrum analysis is performed in the same manner as in Example 1 to obtain the auditory semantic vector. If the input is script text, the text parsing path of Example 1 will be used.

[0053] Ultimately, the high-level semantic feature vector Depend on , With text vectors (If present) Generated via a cross-modal fusion network: , , , Modal weights can be manually adjusted via the user interface to emphasize visual, auditory, or textual cues.

[0054] Secondly, the emotional evolution state machine module introduces a video rhythm synchronization mechanism. The system additionally analyzes the frequency of shot transitions and motion intensity in the video to generate rhythm-driven signals. The rhythm-driven signal is injected into the emotional state transition function as an additional input: When the camera switches quickly ( When approaching 1), the emotional state update step size automatically increases, making the emotional changes more intense and matching the tension of the scene; conversely, slow motion ( A value close to 0 tends to stabilize the emotional state. Furthermore, the target emotion guidance vector... No longer relying solely on preset templates, but rather determined by a small LSTM network based on... It predicts the current scene content in real time, enabling more granular control of emotional trajectory.

[0055] Third, the hierarchical controllable generation network module is optimized into a streaming generation mode. To meet real-time requirements, the generator adopts a non-autoregressive architecture: it predicts the note sketches for the next 8 time steps at once, and then refines them iteratively.

[0056] The style code F is no longer statically queried, but dynamically adjusted by the style tracker based on the acoustic characteristics of the currently generated segment, ensuring that the style transitions naturally with the development of the plot (such as from suspense to warmth).

[0057] The rejection window of the diversity injection unit is shortened to 4 time steps to accommodate the needs of short-duration music, while introducing a "theme motif" memory mechanism: if the user marks a melody as the core motif, the system will repeat its variants with a low probability in subsequent generation to enhance the musical narrative.

[0058] Finally, the post-processing module adds a dynamic tonality adaptation function. The system analyzes the dominant color tone of the scene in real time (e.g., cool colors correspond to minor keys, warm colors correspond to major keys). If a conflict between the tonality and the mood of the scene is detected, the system will forcibly switch the key center during the harmony optimization stage. For example, when the scene changes from dark to bright, the system will change the tonality from minor to relative major at the beginning of the next musical phrase and ensure a smooth transition through a common tone.

[0059] Through the aforementioned enhancements, this embodiment enables the system to efficiently serve film and television creation scenarios, ensuring music quality while achieving deep emotional resonance and rhythmic synchronization with visual content, significantly improving the creative efficiency of professional users.

[0060] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0061] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. An artificial intelligence-based automatic music melody generation system, characterized in that, include: The intent parsing and feature mapping module is used to receive and parse the creative intent information in the form of text, tags or audio input by the user, and transform it into a structured high-level semantic feature vector. The emotional evolution state machine module is used to initialize the emotional state vector based on the high-level semantic feature vector, and dynamically and sequentially update the emotional state vector during the melody generation process according to the preset emotional evolution dynamic model, so as to drive the emotional direction of the melody. The hierarchical controllable generative network module is used to receive the real-time emotional state vector output by the emotional evolution state machine module, and based on this vector and the high-level semantic feature vector, generate a note sequence with specific style, emotional coherence and diversity through a multi-layer neural network architecture including a style decoupling encoder, an emotional conditionalization generator and a diversity injection unit. The post-processing and music theory constraint module is used to apply music theory-based hard rule constraints and harmony optimization processing to the original note sequence output by the hierarchical controllable generation network module, so as to ensure the compliance of the generated melody with music theory and the harmony of hearing.

2. The artificial intelligence-based automatic music melody generation system according to claim 1, characterized in that, The parsing process of the intent parsing and feature mapping module is as follows: When the user inputs text, a pre-trained language model is used to extract the semantic embedding of the text, and a fully connected mapping network is used to project it onto a high-dimensional semantic feature vector aligned with the music feature space. When the user input is audio, the Mel-spectral features of the audio are extracted and encoded into the high-level semantic feature vector by a convolutional neural network encoder; When the user inputs style, mood, or scene tags, the discrete tags are mapped to continuous semantic feature vectors through a learnable tag embedding table.

3. The artificial intelligence-based automatic music melody generation system according to claim 2, characterized in that, The emotion evolution state machine module includes an emotion state vector register and an emotion state transition function; The emotional state vector register is used to store the emotional state vector at the current moment. The emotional state vector is a multi-dimensional real number vector, and its different dimensions represent different aspects of the emotion. The emotional state transition function is a parameterized recurrent neural network. Its inputs are the emotional state vector of the previous time step, the target emotional guidance vector derived from the high-level semantic feature vector at the current time step, and the learnable emotional evolution inertia coefficient. The emotional state transition function calculates and outputs the updated emotional state vector at the current time step.

4. The artificial intelligence-based automatic music melody generation system according to claim 3, characterized in that, The style decoupling encoder in the hierarchical controllable generative network module is used to decouple the hidden layer representation of the basic generative model pre-trained on a large and diverse music dataset. This style decoupling encoder learns to separate musical representations into a style-independent content latent space and a style-dependent style latent space through adversarial training. The style decoupled encoder receives the intermediate representation of the underlying generative model as input and outputs the decoupled content encoding and style encoding.

5. The artificial intelligence-based automatic music melody generation system according to claim 4, characterized in that, The sentiment conditional generator in the hierarchical controllable generative network module is based on a sequence-to-sequence model with an attention mechanism. The decoder part of the emotion conditionalization generator, when generating notes at each time step, has its initial hidden state conditionalized by the content encoding, the style encoding, and the emotion state vector provided by the emotion evolution state machine module at the current moment. The emotional state vector is weighted and fused with the content-style joint encoding through a conditional fusion gating mechanism to form the conditional context vector of the decoder.

6. The artificial intelligence-based automatic music melody generation system according to claim 5, characterized in that, The diversity injection unit in the hierarchical controllable generation network module is integrated into the sampling process of the emotion conditionalization generator; This diversity injection unit introduces a controllable temperature parameter and an exclusionary penalty term based on historically generated segments after the decoder outputs the probability distribution of the next note; The temperature parameter is used to adjust the smoothness of the probability distribution, and the exclusion penalty term is used to reduce the selection probability of frequently occurring note or interval combinations in recently generated note sequences.

7. The artificial intelligence-based automatic music melody generation system according to claim 6, characterized in that, The specific processing performed by the post-processing and music theory constraint module includes pitch range limiting, rhythmic time value quantization, prohibition of illegal interval continuity, and harmony optimization. The harmony optimization is achieved through a lightweight harmony analysis submodule. This submodule performs real-time harmony analysis on the generated melody fragments, identifies their implicit chord progressions, and automatically adjusts the pitch of individual notes to the minimum extent to correct harmony errors when it detects that the chords do not conform to the preset harmony rules.

8. The artificial intelligence-based automatic music melody generation system according to claim 7, characterized in that, The system operates in an iterative, loop-based manner, and its workflow is as follows: The intent parsing and feature mapping module parses the user input, outputs a high-level semantic feature vector, and sends it to the emotion evolution state machine module for initialization. The emotional evolution state machine module outputs a series of emotional state vectors in time step sequence according to the emotional evolution dynamics model. The hierarchical controllable generative network module receives these temporal emotional state vectors and the high-level semantic feature vectors, and gradually generates the corresponding musical note sequence; The post-processing and music theory constraint module performs compliance correction and optimization on the complete note sequence, and outputs the final music melody data.

9. The artificial intelligence-based automatic music melody generation system according to claim 8, characterized in that, The emotional evolution dynamics model supports a variety of preset emotional evolution templates, including linear incremental type, wave oscillation type, conflict resolution type and stable maintenance type; Users can select an appropriate emotion evolution template through intent input or the system can automatically match it based on the high-level semantic features. The emotion evolution template affects the calculation logic of the target emotion guidance vector and the emotion evolution inertia coefficient in the emotion state transition function in the form of parameters.

10. The artificial intelligence-based automatic music melody generation system according to claim 9, characterized in that, The training process of the system is divided into three stages: In the first stage, the basic generative model and the style decoupled encoder were pre-trained using a large-scale unlabeled music dataset; In the second stage, music datasets with style and emotion labels were used to jointly fine-tune the emotion conditionalization generator, the emotion evolution state machine module, and the intent parsing and feature mapping module. In the third stage, a reinforcement learning strategy based on music theory rules and diversity evaluation indicators is introduced to further optimize the parameters of the diversity injection unit and some parameters of the generator.

Images