Symbol music generation method based on texture perception and adaptive representation alignment

Abstract

The invention discloses a symbol music generation method based on texture perception and adaptive representation alignment. The method is characterized by comprising the steps of conditional decoupling and extraction of multi-scale music element features, conditional coding, texture perception FLUX music model construction, adaptive representation alignment, MIDI file generation and the like. Compared with the prior art, the method has the advantages that the structural advantage and multi-scale information of the symbol music are fully utilized, the texture consistency, chord accuracy and style controllability of the generated music are remarkably improved, the training and reasoning cost is reduced, and the generalization ability and practicability of the model are further improved. Through the three-in-one design of'conditional decoupling, texture perception diffusion and self-adaptive expression alignment ', the problems of semantic loss, inconsistent texture, uncontrollable chord and the like caused by neglecting of internal association of music in a traditional method are effectively solved, and the method is efficient, reliable and easy to implement and has good application prospects in the fields of automatic composition, interactive music creation and the like.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of symbolic music generation, and particularly to a symbolic music generation method based on texture perception and adaptive representation alignment. BACKGROUND

[0002] Current artificial intelligence music creation research is mostly focused on audio music, because the technical implementation of audio music is relatively intuitive, and it can quickly adapt to market demand. Although significant progress has been made in the field of artificial intelligence-generated audio music, this approach, which only considers music as audio, does not integrate professional knowledge in the field of music well, making it difficult to further fine-tune and optimize the music created by artificial intelligence in the actual music production process. In contrast, symbolic music provides a more comprehensive and structured way to understand and express music, with good interpretability, making music generation and editing more intuitive and controllable.

[0003] In the automatic generation of symbolic music, a prominent problem is the insufficient or ineffective constraint of guidance conditions. To solve this problem, researchers have introduced multi-level condition guidance methods and achieved some success in the field of symbolic music generation. However, these methods do not adequately explore the interaction between different levels of conditions, leading to potential problems such as deviation from expected guidance, lack of musical coherence, or insufficient stylistic consistency in generated music. To further improve the quality of symbolic music generation, it is necessary to conduct more in-depth research on the interaction between different levels of conditions and their impact on music creation. For example, the relationship between melody and chord, the coordination of rhythm and harmony, the matching of musical structure and emotional expression, and other factors are all key factors that affect the overall effect of music. By better understanding and decoupling these interactions, the music generation process can be more effectively guided, resulting in more natural and artistic generated music.

[0004] In summary, the existing music automatic generation methods have made significant progress in whole-music hierarchical generation, but still have some limitations. For example, relying only on melody or chord as generation conditions ignores the inherent connection between them that can interact with each other. This monotonous condition form may result in a lack of melody quality or inaccurate style control. In addition, since whole-music hierarchical generation usually uses improved but still relatively traditional diffusion models, they lack flexibility in adjusting condition information at different scales, usually guiding the model in a single way. While detailed conditions can provide more fine-grained composition control, this can also lead to excessive computational burden, making model training difficult and prolonging inference time. On the other hand, overly abstract conceptual conditions may not accurately convey information, resulting in insufficient guidance and ineffective coupling. SUMMARY

[0005] The purpose of the present application is to propose a symbolic music generation method based on texture perception and adaptive representation alignment to overcome the shortcomings of the prior art. The method adopts a three-in-one design mixed FLUX diffusion model of "conditional decoupling + texture perception diffusion + adaptive representation alignment", injects melody and chord representation at different stages respectively to realize fine generation control, then introduces an adaptive representation alignment framework based on soft voting mechanism, dynamically selects the optimal intermediate layer features and true value for semantic constraint, and finally fuses the flow matching loss to complete end-to-end training. Through the three-in-one design of "conditional decoupling, texture perception diffusion and adaptive representation alignment", the present application effectively overcomes the problems of semantic loss, texture inconsistency and uncontrollable chord caused by ignoring the internal correlation of music in traditional methods, fully utilizes the structured advantages and multi-scale information of symbolic music, significantly improves the texture consistency, chord accuracy and style controllability of generated music, reduces the training and inference cost, and improves the model generalization ability and practicality. The method decouples the input into two main music elements of melody and chord progression, uses a pre-trained model to embed and encode the two elements, efficiently uses the guided conditional constraint diffusion model to extract the intermediate representation of the target layer during the diffusion process, calculates the cosine similarity of the representation alignment framework and the chord progression true value, and selects the maximum and second maximum values by the soft voting mechanism to guide the generation, which greatly improves the style consistency of the generated music and better matches the user's generation needs. The method is simple, practical, efficient and reliable, easy to implement, and has good application prospects in automatic composition, interactive music creation and other fields.

[0006] The purpose of the present application is achieved by a symbolic music generation method based on texture perception and adaptive representation alignment, characterized in that the method first decouples the input melody and chord under the condition, extracts multi-scale music element features, then uses a double / single stream mixed FLUX diffusion model to inject melody and chord representation at different stages respectively to realize fine generation control, and introduces an adaptive representation alignment framework based on soft voting mechanism to dynamically select the optimal intermediate layer features and true value for semantic constraint, and finally fuses the flow matching loss to complete end-to-end training. The symbolic music generation specifically includes the following steps: Model input The model input is guided by the condition Lead sheet, which is a MIDI format music draft, and The input MIDI format file is used as a condition, and the content is a multi-track music draft.

[0007] Conditional decoupling The melody and chord progression are decoupled from the guided condition Lead sheet, and a rule-based conditional decoupling method is used to separate and supplement the melody and chord progression in the Lead sheet.

[0008] Conditional encoding The melody and chord progression obtained by decoupling are respectively conditionally encoded, and the pre-trained encoding model is used to encode The melody and chord progression obtained by decoupling are encoded to obtain a feature embedding representation of the FLUX model prompt condition.

[0009] Guided music generation The encoded conditions are input into the texture-aware FLUX music model, so that the diffusion process of the model is guided to generate music, and the diffusion process is constrained according to the input feature embedding representation, and a complete music is gradually generated through a loop denoising process from a Gaussian distribution as a starting point for generation.

[0010] Representation constraint The chord progression representation in the diffusion process is constrained, specifically including: 5.1: The music model generates music under the representation constraint framework, and the diffusion intermediate representation of a predetermined number of layers is aligned with the previously separated chord progression embedding representation in terms of cosine similarity, so that the chord progression of the generated music meets the condition requirements; 5.2: Sort the cosine similarity of all aligned layer representations and the target chord progression embedding representation, and select the maximum value and the second maximum value as the alignment result, and return the alignment result to the model for further adjustment of the generation process.

[0011] 6) Conversion and output of format files The final result of the FLUX model, i.e. the generated music, is converted into a MIDI format music file and output to the user.

[0012] Compared with the prior art, the present application has the following beneficial technical effects and significant technical progress: 1) The present application proposes a symbolic music generation guide method based on conditional decoupling, which uses decoupling of input conditions to more finely constrain the generation process, and achieves high consistency between generated music and guide condition style; 2) The present application proposes a representation alignment method of soft voting mechanism, which improves the robustness of the alignment framework, and significantly improves the control of the condition on the generated music, and obtains more accurate and natural generation results; 3) Effectively overcomes the problems of semantic loss, texture inconsistency and uncontrollable chords caused by ignoring the internal correlation of music, fully utilizes the structured advantages and multi-scale information of symbolic music, significantly improves the texture consistency, chord accuracy and style controllability of the generated music, reduces the training and inference cost, and improves the model generalization ability and practicality.

[0013] 4) The method is simple, practical, efficient and reliable, easy to implement, and has good application prospects in automatic composition, interactive music creation and other fields. BRIEF DESCRIPTION OF DRAWINGS

[0014] Figure 1 The flowchart of the present application; Figure 2 is a specific operation schematic diagram of Example 1. DETAILED DESCRIPTION

[0015] The purpose of the present application is to overcome the shortcomings of the prior art and provide a symbolic music generation method based on texture perception and adaptive representation alignment, characterized in that the method first decouples the input main melody and chord under certain conditions, extracts multi-scale music element features; then through the FLUX diffusion model of double / single stream mixing, melody and chord representation are injected at different stages respectively to realize fine generation control; then an adaptive representation alignment framework based on soft voting mechanism is introduced, the optimal intermediate layer features and true value are dynamically selected for semantic constraint, and then stream matching loss is fused to complete end-to-end training. Through the trinity design of "conditional decoupling + texture perception diffusion + adaptive representation alignment", the present application effectively overcomes the problems of semantic loss, texture inconsistency and uncontrollable chord caused by ignoring the internal correlation of music in traditional methods, fully utilizes the structured advantages and multi-scale information of symbolic music, significantly improves the texture consistency, chord accuracy and style controllability of generated music, reduces the training and inference cost, and improves the model generalization ability and practicality.

[0016] Referring to Figure 1 , the present application specifically comprises the following steps: Lead sheet is input as a model guide condition, and the lead sheet is a MIDI format music draft 1.1: Input the MIDI format file as a condition for the model, and the content is a multi-track music draft.

[0017] Decouple the melody and chord progression from the guide condition Lead sheet 2.1: Use a rule-based conditional decoupling method to separate and supplement the melody and chord progression in the Lead sheet.

[0018] Conditionally encode the decoupled melody and chord progression 3.1: Use a pre-trained encoding model to encode the decoupled melody and chord progression to obtain feature embedding representation of the FLUX model prompt condition.

[0019] Guided music generation The encoded conditions are input into a texture-aware FLUX music model, which generates music by guiding the diffusion process of the model with the conditions, and the FLUX model constrains the diffusion process according to the input feature embedding representation, and generates complete music step by step through a process of cycle denoising from a Gaussian distribution as the starting point of generation.

[0020] Characterize the chord progression in the diffusion process 5.1: The model generates music under the framework of characterization constraints, and aligns the diffusion intermediate representations of a predetermined number of layers with the previously separated chord progression embedding representation to constrain the chord progression of the generated music to meet the condition requirements. 5.2: Sort the cosine similarity of all aligned layer representations with the target chord progression embedding representation, and select the maximum and second maximum as the alignment result, and return the alignment result to the model for further adjustment of the generation process.

[0021] 6) Convert the final result of the model into a MIDI format file and output 6.1: Convert the output of the FLUX model into a MIDI format music file and return it to the user.

[0022] The application will be further described below in conjunction with specific examples and drawings.

[0023] Example 1 Referring to Figure 2 , the following steps are used to generate a simple music segment: Step 1: input music in MIDI format as conditions, and separate the melody and chord progression from the input using a rule-based decoupling method; Step 2: use a pre-trained model to encode the chord progression and melody respectively; Step 3: input the encoding into a texture-aware FLUX music model, and the model iteratively denoises from a Gaussian noise starting point and constrains the denoising process with the input encoding; Step 4: extract the chord representation of the target layer in the denoising process, calculate the cosine similarity with the true value, and use a soft voting-based selection method to obtain the maximum and second maximum as the constraint target optimization generation; Step 5: convert the output of the model into a MIDI format file and return it to the user.

[0024] The above specific implementation is only to further illustrate the application, and is not used to limit the application patent. Any equivalent implementation of the application should be included in the scope of the claims of the application patent.

Claims

1. A symbolic music generation method based on texture perception and adaptive representation alignment, characterized in that, The diffusion process of the FLUX music model is guided by the input conditions and the representation alignment method, and the symbolic music generated is consistent with the style of the guiding conditions, specifically including: Step 1: input the guiding condition Lead sheet into the FLUX music model, and the Lead Sheet is a MIDI format music draft; Step 2: decouple the melody and chord progression from the guiding condition Lead sheet; Step 3: conditionally encode the melody and chord progression obtained by decoupling; Step 4: input the encoded conditions into the texture-aware FLUX music model to guide the diffusion process of the model and generate music; Step 5: constrain the chord progression representation in the diffusion process; Step 6: convert the final result of the FLUX music model, i.e. the generated music, into a MIDI format file and output.

2. The method of claim 1, wherein, In step 1, the FLUX music model takes the input MIDI format file as a condition, which requires the content to be a multi-track music draft.

3. The method of claim 1, wherein, In step 2, the rule-based condition decoupling method is used to separate and supplement the melody and chord progression in the Lead sheet.

4. The method of claim 1, wherein, In step 3, the pre-trained encoding model is used to encode the melody and chord progression obtained by decoupling to obtain the feature embedding representation of the FLUX music model prompt condition.

5. The method of claim 1, wherein, In step 4, the FLUX music model constrains the diffusion process according to the input feature embedding representation and generates complete music through the process of loop denoising from the Gaussian distribution starting point.

6. The method of claim 1, wherein, Step 5 specifically includes: 5.1: The FLUX music model generates music under the representation constraint framework, aligns the embedding representation of the chord progression separated in advance with the intermediate representation of a predetermined number of diffusion layers in terms of cosine similarity, to constrain the chord progression of the generated music to meet the condition requirements; 5.2: Sort the cosine similarity of all aligned layer representations and target chord progression embedding representations, select the maximum and second maximum values as the alignment results, and return the alignment results to the FLUX music model for further adjustment of the generation process.

7. The method of claim 1, wherein, Step 6 converts the output of the FLUX music model into a MIDI format music file and returns it to the user.

Images