AI music creation information processing method based on reinforcement learning

Abstract

The invention discloses an AI music creation information processing method based on reinforcement learning, and relates to the technical field of music information processing, and the method comprises the steps: inputting a macroscopic music structure into a dynamic music knowledge map, searching an optimal element path through a map traversal and path search algorithm, and generating a symbolized action sequence in combination with rhythm-type suggestions and adapter prompts; the symbolized action sequence is decoded into a specific note sequence through a Transform decoder, smooth transition between notes is carried out in combination with an attention mechanism, and complete music data is generated according to real-time context fine tuning generation parameters; and based on the complete music data, in combination with a music theory rule, performing multi-modal music quality evaluation to generate a reward signal, and in combination with a historical hot music database, calculating style similarity and a market potential index to generate a multi-target reward function. According to the invention, intelligent selection and combination of music elements are realized, and the narrative integrating degree and professional expressive force of music are improved.

Description

Technical Field

[0001] This invention relates to the field of music information processing technology, and in particular to an AI-based music creation information processing method based on reinforcement learning. Background Technology

[0002] In recent years, artificial intelligence technology has made progress in the field of music creation, especially with automatic composition systems based on deep learning and reinforcement learning becoming a research hotspot. With the development of multimodal fusion technology, existing systems can now combine audio, text, and visual features to generate music. In film and television scoring applications, the industry has begun to explore combining plot analysis with music generation, using sentiment analysis methods to identify the emotional curves of scripts and generating corresponding emotional musical fragments based on rule-based systems. Recent research shows that reinforcement learning exhibits unique advantages in music generation tasks; by designing reward functions to guide model optimization, it can produce more artistic and coherent musical works. Knowledge graph technology further enhances the logical consistency of music generation, enabling systems to better understand music theory rules and stylistic features.

[0003] Existing technologies for music generation suffer from two key shortcomings: first, they lack in-depth utilization of the emotional content of the storyline, resulting in generated music that lacks emotional resonance with the narrative and struggles to accurately capture plot climaxes and emotional turning points; second, they lack a multi-objective optimization mechanism, failing to simultaneously consider musical theoretical quality, stylistic similarity, and market potential, thus limiting the artistic and commercial value of the generated music. These shortcomings severely restrict the application depth of AI music generation technology in professional fields such as film and television scoring, and fail to meet the industry's actual demand for high-quality contextual music. Summary of the Invention

[0004] In view of the aforementioned existing problems, the present invention is proposed.

[0005] Therefore, this invention provides a reinforcement learning-based AI music creation information processing method that solves the problems of insufficient depth matching of plot and emotion and lack of multi-objective optimization in AI music generation.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] This invention provides a reinforcement learning-based AI method for processing music composition information, comprising:

[0008] We collect semantic vector data of the plot development trajectory, emotional change curve and multimodal feature fusion of the series, and use LSTM-based policy network to identify the climax and emotional turning point of the plot to obtain the macro music structure.

[0009] The macroscopic music structure is input into a dynamic music knowledge graph. The optimal element path is searched through graph traversal and path search algorithms. Combined with rhythmic suggestions and orchestration cues, symbolic action sequences are generated.

[0010] The symbolic action sequence is decoded into a specific note sequence by the Transformer decoder, and the smooth transition between notes is achieved by combining the attention mechanism. The generation parameters are fine-tuned according to the real-time context to generate complete music data.

[0011] Based on complete music data and combined with music theory rules, multimodal music quality assessment is performed to generate reward signals. Furthermore, style similarity and market potential index are calculated by combining historical popular music databases to generate a multi-objective reward function.

[0012] Based on the multi-objective reward function, the policy networks of the high-level manager and the low-level executor are updated simultaneously through the proximal policy optimization algorithm, and the final complete music data and music creation analysis report are generated.

[0013] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method described in this invention, the method employs an LSTM-based policy network to identify plot climaxes and emotional turning points to obtain the macroscopic music structure. The specific steps are as follows:

[0014] Semantic vector data is input into an LSTM-based policy network, and the hidden state and output probability distribution at each time step are obtained through forward propagation of the network to generate an enhanced temporal feature representation.

[0015] Based on temporal features, a list of plot climaxes and emotional turning points is identified through peak detection and change point detection.

[0016] By combining the list of plot climaxes and emotional turning points with the time distribution, the boundaries of musical segments are divided and segment types are assigned to generate a macroscopic musical structure.

[0017] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method of the present invention, the steps of inputting the macroscopic music structure into a dynamic music knowledge graph and searching for the optimal element path through graph traversal and path search algorithms are as follows:

[0018] Input the macroscopic music structure into the dynamic music knowledge graph, perform multi-element analysis, and generate a set of graph query commands;

[0019] Based on the set of graph query instructions, the optimal element path is generated by graph traversal and path search, and a set of candidate music element paths is generated.

[0020] As a preferred embodiment of the reinforcement learning-based AI music composition information processing method of the present invention, the step of generating a symbolic action sequence by combining rhythmic suggestions and orchestration cues includes the following specific steps.

[0021] Based on the candidate music element path set, the rhythmic characteristics of music concept nodes are identified through multi-level analysis methods to generate rhythmic suggestions;

[0022] By combining rhythmic suggestions with an instrument timbre database using a multi-dimensional dynamic mapping method, orchestration prompts are generated.

[0023] The candidate music element path set, rhythm pattern suggestions, and orchestration cues are integrated to generate a symbolic action sequence.

[0024] As a preferred embodiment of the reinforcement learning-based AI music composition information processing method of the present invention, the step of decoding the symbolic action sequence into a specific note sequence using a Transformer decoder and combining it with an attention mechanism to achieve smooth transitions between notes includes the following specific steps.

[0025] The symbolic action sequence is converted into readable sequence data, and the sequence data is decoded into a specific note sequence by the Transformer decoder;

[0026] Based on the specific note sequence, an attention mechanism is used to optimize the smooth transition between notes, generating an optimized note sequence.

[0027] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method of the present invention, the specific steps for generating complete music data by fine-tuning the generation parameters based on real-time context are as follows:

[0028] Based on the optimized note sequence, the pitch, duration, and velocity parameters are fine-tuned in real time context to obtain the set of fine-tuned note parameters;

[0029] Based on the finely tuned set of note parameters, complete music data is generated through dynamic multitrack synthesis and real-time audio rendering.

[0030] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method of the present invention, the specific steps for generating a reward signal by performing multimodal music quality assessment based on complete music data and in conjunction with music theory rules are as follows:

[0031] Acoustic features are extracted from complete music data using a multimodal audio feature extraction method. The acoustic features are then standardized to generate a set of standardized feature vectors.

[0032] Multimodal music quality assessment is performed on a standardized feature vector set, and a reward signal is generated by combining music theory rules.

[0033] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method of the present invention, the step of combining a historical popular song database to calculate style similarity and market potential index to generate a multi-objective reward function includes the following specific steps.

[0034] Based on the reward signals, and combined with the feature set of popular songs in the historical hot song database, the style similarity and market potential index are calculated.

[0035] By integrating market potential indices and reward signals, a multi-objective reward function is generated.

[0036] As a preferred embodiment of the reinforcement learning-based AI music creation information processing method of the present invention, the specific steps of updating the policy networks of the high-level manager and the low-level executor simultaneously according to the multi-objective reward function and the proximal policy optimization algorithm are as follows:

[0037] The multidimensional reward function is analyzed using a proximal policy optimization algorithm to generate standardized reward data.

[0038] Based on standardized reward data, the policy network update gradient is obtained through a near-end policy optimization algorithm, and the policy network update gradient vector is generated.

[0039] The gradient vector is updated based on the policy network, thus updating the parameters of the policy networks of the high-level manager and the low-level executor.

[0040] As a preferred embodiment of the AI ​​music creation information processing method based on reinforcement learning described in this invention, the final complete music data and music creation analysis report are generated through a multi-level music generation and intelligent analysis method based on the updated policy network of the high-level manager and the low-level executor.

[0041] The beneficial effects of this invention are as follows: by inputting semantic vector data into an LSTM-based policy network, accurate identification of plot climaxes and emotional turning points is achieved, enhancing the emotional expressiveness and narrative quality of the generated music; through graph traversal and path search algorithms of a dynamic music knowledge graph, intelligent selection and combination of musical elements are achieved, improving the narrative fit and professional expressiveness of the music. Attached Figure Description

[0042] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 A flowchart of an AI-based music creation information processing method for reinforcement learning.

[0044] Figure 2 A flowchart for multimodal data processing and macroscopic music structure generation.

[0045] Figure 3 A flowchart for generating dynamic music knowledge graphs and symbolic action sequences.

[0046] Figure 4 A flowchart for music sequence decoding and optimization. Detailed Implementation

[0047] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0048] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0049] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The phrase "in one embodiment" appearing in different places in this specification does not necessarily refer to the same embodiment, nor is it a single or selective embodiment that is mutually exclusive with other embodiments.

[0050] Reference Figures 1-4 This is one embodiment of the present invention, which provides an AI music creation information processing method based on reinforcement learning, comprising the following steps:

[0051] S1: Collect the plot development trajectory, emotional change curve and semantic vector data after multimodal feature fusion of the series, and use the LSTM-based policy network to identify the climax and emotional turning point of the plot to obtain the macro music structure;

[0052] S1.1: Collect the plot development trajectory, emotional change curves, and multimodal features of the TV series, and fuse them into semantic vector data;

[0053] Specifically, the plot development trajectory data of the series is obtained by acquiring video scene switching points, camera movement types, and key event timestamps (such as plot turning point markers); visual emotion scores are generated by processing video keyframes through facial expression recognition; emotion polarity values ​​are obtained by applying emotion analysis natural language processing methods to output dialogue text; and multidimensional emotion vectors are obtained by extracting background music spectral features using audio signal processing tools, generating emotion change curves; multimodal features include visual color distribution features and audio Mel-frequency cepstral coefficient features; time axis alignment is performed through a cross-modal attention mechanism, and a deep neural network encoder is used to map the aligned multimodal features to a unified semantic space, and dimensionality-reduced semantic vector data is generated through feature compression methods.

[0054] S1.2: Input semantic vector data into an LSTM-based policy network, obtain the hidden state and output probability distribution of each time step through network forward propagation, and generate an enhanced temporal feature representation;

[0055] Specifically, semantic vector data is input into an LSTM-based policy network in time-step order. The LSTM-based policy network uses a sigmoid function at the input gate to control the degree of fusion of current input information, a sigmoid function at the forget gate to adjust the retention ratio of historical information, and a sigmoid function at the output gate to control the output strength of the hidden state. At each time step, a tanh activation function is used to generate candidate cell states and update the cell states. Through forward propagation, the hidden state and output probability distribution are obtained step-by-step (generated using a softmax function), generating an enhanced temporal feature representation. This enhanced temporal feature representation serves both the high-level manager and the low-level executor policy networks: the high-level manager policy network uses the temporal feature representation to make macroscopic music structure decisions (such as generating paragraph sequences [introductory paragraph - verse - chorus] and duration allocation), while the low-level executor policy network obtains microscopic music elements based on the same temporal feature representation, ensuring the coordinated optimization of macroscopic structure and microscopic music elements through shared temporal feature representation.

[0056] It should be noted that the training process of the LSTM policy network optimizes the network parameters by minimizing the policy loss function (e.g., cross-entropy loss): forward propagation is used to obtain the hidden state and output probability distribution at each time step, and the weight parameter update is obtained step by step based on the loss gradient through the backpropagation algorithm. The gradient flows through the sigmoid function of the output gate, forget gate, and input gate and the corresponding weight parameters (such as the input gate weight matrix). At the same time, gradient pruning method is applied to prevent training instability. The policy network converges to the optimal performance by iteratively adjusting the parameters of the cell state and hidden state.

[0057] S1.3: Based on the temporal feature representation, identify a list of plot climaxes and emotional turning points through peak detection and change point detection;

[0058] Specifically, based on temporal feature representation, a sliding window algorithm is used to obtain local extrema. Potential peak points are identified by setting dynamic judgment criteria. Simultaneously, a Bayesian change point detection algorithm is used to obtain changes in sequence distribution features. When the cumulative probability exceeds a fixed probability standard (e.g., 0.8), it is marked as a change point. The detected peak points and change points are then time-aligned and type-labeled (e.g., climaxes are labeled as "peaks," and emotional turning points as "turns"), generating a list of dramatic climaxes and emotional turning points containing timestamps, type identifiers, and intensity values. The entire process ensures detection accuracy through dynamic adjustment and fixed reference values, avoiding subjective bias and improving the fit between music and plot.

[0059] It should be noted that the dynamic judgment criterion identifies statistical peak points by acquiring the mean and standard deviation of the time-series characteristic data within the sliding window, and using the mean combined with a dynamic benchmark (e.g., 1.5 times the standard deviation). The dynamic judgment criterion is dynamically adjusted according to the data distribution characteristics, effectively distinguishing noise from true signals. The setting of the dynamic benchmark fully considers the data fluctuation characteristics, and the sensitivity is dynamically adjusted by the statistics within the sliding window, which ensures the reliability of peak detection while avoiding the oversensitivity or omission problems that may be caused by a fixed standard.

[0060] S1.4: Combine the list of plot climaxes and emotional turning points with the time distribution, divide the boundaries of music segments and assign segment types to generate a macro music structure.

[0061] Furthermore, by precisely aligning and matching the list of plot climaxes and emotional turning points (containing sequences of timestamps, type identifiers, and degrees of change) with the time distribution, a sliding window algorithm (e.g., a window length of 30 seconds) is used to identify the density distribution curve of climaxes and the frequency distribution map of turning points. The boundaries of musical segments are divided according to the troughs of the density distribution curve of climaxes. The segment type is assigned based on the average intensity value of climaxes and the degree of change of turning points within the segment, combined with music theory rules: (the area with climax intensity greater than 0.8 and degree of change greater than 0.6 is assigned as the chorus segment; the area with climax intensity between 0.4 and 0.8 and degree of change between -0.2 and 0.2 is assigned as the verse segment; and the area with degree of change of turning points less than -0.5 is assigned as the bridge segment). At the same time, the default settings of the intro and outro segments are considered to generate a structured macroscopic musical structure sequence, ensuring that the musical segments are highly synchronized with the plot and emotions.

[0062] It should be noted that the music theory rules are a set of standardized principles built upon the foundation of traditional music theory, including principles of harmony, rhythmic organization methods, scale structure systems, and instrument orchestration methods. The music theory rules are extracted and encoded from academic literature on music theory and historical musical works to form constraints, which are then integrated into a dynamic music knowledge graph as a benchmark for element association and path search, ensuring that musical creations conform to professional artistic standards. The process of generating music theory rules involves automatically extracting patterns from a large number of scores and academic literature, and combining machine learning algorithms to optimize rule weights, which are then solidified into relational edges and node attributes in the knowledge graph, supporting real-time querying and dynamic updates.

[0063] S2: Input the macroscopic music structure into the dynamic music knowledge graph, search for the optimal element path through graph traversal and path search algorithms, and generate symbolic action sequences by combining rhythmic suggestions and orchestration cues.

[0064] S2.1: Input the macroscopic music structure into the dynamic music knowledge graph, perform multi-element analysis, and generate a set of graph query instructions;

[0065] Specifically, the macroscopic music structure is input into a dynamic music knowledge graph. A structured multi-element decomposition method is used to obtain multi-element information such as paragraph type, suggested duration, basic emotion type, and intensity level from the macroscopic music structure. Key attributes (such as paragraph sequence order and emotion tags) are extracted. The dynamic music knowledge graph then converts this multi-element information into a standardized query language based on music theory rules and element relationships (such as chord compatibility and rhythmic pattern matching), generating a set of graph query instructions (such as a sequence of SPARQL or Cypher query statements containing paragraph type constraints, duration ranges, and emotion matching conditions). This is used for subsequent graph traversal and path search operations. The entire process ensures that the query instructions accurately correspond to each component of the macroscopic music structure, improving the accuracy and efficiency of music element retrieval.

[0066] The construction of a dynamic music knowledge graph begins with materializing music theory concepts into nodes and abstracting relationships into edges. Core music element nodes and relational edges are defined according to music theory rules. Music element nodes include chord entities, rhythmic nodes, scale nodes, and instrument nodes. Relationship edges encompass compatibility relationships, transitional relationships, emotional connections, and stylistic connections. During construction, based on music theory rules, nodes and edges are structurally stored using graph database methods, forming a queryable knowledge graph architecture that supports multimodal music feature mapping and dynamic update mechanisms.

[0067] S2.2: Based on the set of graph query instructions, the optimal element path is generated by graph traversal and path search to generate a set of candidate music element paths;

[0068] Specifically, based on the graph query command set, the system traverses the music concept nodes and relation edges in the dynamic music knowledge graph. A community detection algorithm (based on the connection patterns between music concept nodes in the dynamic music knowledge graph, identifying highly cohesive node clusters through tight connections; these clusters naturally form based on connection density and music theory rules, thus quickly locating potential path candidate regions, improving graph traversal efficiency, and providing a structured foundation for multi-objective path search) identifies node clustering features and high-density connection regions as potential path candidates. It explores connection patterns between nodes and identifies potential paths. The system combines path search with multi-objective optimization algorithms to balance the artistic and technical aspects of the path, searching for Pareto optimal solutions in parallel. It then searches for optimal element paths (e.g., a sequence from a C major triad to a G dominant seventh chord) and performs real-time compliance checks based on music theory rules to verify path validity, generating a candidate music element path set (containing multiple music element sequences that conform to the query command).

[0069] S2.3: Based on the candidate music element path set, identify the rhythmic characteristics of music concept nodes through multi-level analysis methods and generate rhythmic suggestions;

[0070] Specifically, the duration distribution characteristics and repetition patterns of rhythmic nodes are extracted through multi-level analysis methods (e.g., detecting the frequency of rhythmic repetition intervals), the rhythmic features of musical concept nodes are identified (e.g., the rhythmic stability index ranges from 0 to 1, and the rhythmic complexity score is based on the frequency of changes in note value combinations), and finally, rhythmic suggestions are generated (e.g., a stable 4 / 4 time signature rhythm is used for the verse, and a syncopated rhythm is used for the chorus).

[0071] S2.4: By combining rhythmic suggestions with an instrument timbre database through a multi-dimensional dynamic mapping method, orchestration prompts are generated;

[0072] Specifically, a multi-dimensional dynamic mapping method is used to combine rhythm suggestions with an instrument timbre database (including instrument types such as piano and strings, timbre features such as brightness range 0-1, and playing techniques such as legato and staccato) to achieve a three-dimensional combination: matching the emotional intensity of the rhythm in the emotional dimension, associating the rhythm pattern with the instrument style tag in the style dimension, and coordinating the tempo with the feasibility of the playing technique in the performance dimension; and dynamically adjusting the priority of instrument combinations through a multi-attribute weighted decision algorithm (e.g., determining the weight of each dimension based on the entropy weight method and performing a weighted summation to synthesize the priority score) to generate orchestration prompts.

[0073] It should be noted that the instrument timbre database is a structured data collection used to store and manage detailed instrument attribute information involved in music creation, including instrument type, timbre characteristic parameters, performance technique descriptions, and dynamic range control data. The instrument timbre database organizes data in a standardized format, supports multi-dimensional queries and matching (such as based on emotional intensity and style tags), and performs real-time retrieval, providing accurate timbre references and combination basis for music generation. The generation process involves extracting instrument type, timbre characteristics, and performance technique description parameters from historical musical works and professional timbre libraries, and ensuring consistency and usability through automated data cleaning and normalization processing.

[0074] S2.5: Integrate the candidate music element path set, rhythm pattern suggestions, and orchestration cues to generate a symbolic action sequence.

[0075] Furthermore, the process of integrating the candidate musical element path set, rhythm suggestions, and orchestration cues using the timeline alignment algorithm is as follows: A dynamic time warping algorithm is used to synchronize the timestamps of all elements with a precision of 10 milliseconds to establish a unified time reference system; and a hierarchical fusion mechanism is used to perform multi-dimensional matching between the chord sequences in the candidate musical element path set, the beat patterns in the rhythm suggestions, and the instrument parameters in the orchestration cues. At the same time, the musical elements are converted into unified symbolic representations, and the attributes of the dynamically fused elements are dynamically allocated with multi-dimensional weights to generate a structured symbolic action sequence.

[0076] S3: The symbolic action sequence is decoded into a specific note sequence through the Transformer decoder, and the attention mechanism is used to make smooth transitions between notes. The generation parameters are fine-tuned according to the real-time context to generate complete music data.

[0077] S3.1: Convert symbolic action sequences into readable sequence data, and decode the sequence data into specific note sequences using a Transformer decoder;

[0078] Specifically, the Transformer decoder receives symbolic action sequences, utilizes a multi-head self-attention mechanism to obtain the dependencies between elements in the sequence, generates hidden state representations through a feedforward neural network, and uses the softmax function of the output layer to decode them into readable sequence data (such as MIDI event sequences including note on / off events). The attention mechanism is then used to optimize the smooth transitions between notes, and the generation parameters (such as note velocity range 0-127, timbre brightness parameters) are fine-tuned based on real-time context parameters (such as emotional intensity value 0.8). The specific note sequence is then output (such as [C4 pitch, quarter note duration, velocity 80, E4 pitch, eighth note duration, velocity 60]). The entire process ensures the accurate conversion of musical elements and artistic fluency.

[0079] S3.2: Based on the specific note sequence, optimize the smooth transition between notes using an attention mechanism to generate an optimized note sequence;

[0080] Specifically, an attention mechanism is used to obtain the correlation between each note in the note sequence. Attention weights are obtained through queries, key-value vectors, and scores are assigned based on these weights to optimize the smooth transitions between notes. This includes adjusting pitch intervals to avoid excessive leaps, balancing duration distribution to ensure rhythmic continuity, and balancing dynamic changes to enhance dynamic naturalness, resulting in an optimized note sequence.

[0081] S3.3: Based on the optimized note sequence, the pitch, duration and velocity parameters are fine-tuned in real time context to obtain the fine-tuned note parameter set, and complete music data is generated through dynamic multi-track synthesis method and real-time audio rendering.

[0082] Specifically, pitch, duration, and velocity parameters are fine-tuned through real-time context: pitch is adjusted via semitone shift, duration via percentage increase / decrease, and velocity via linear gradation. This fine-tuning process dynamically adjusts pitch, duration, and velocity parameters based on music theory rules, obtaining a set of fine-tuned note parameters to ensure it meets real-time composition requirements. Independent instrument tracks are generated using a dynamic multi-track synthesis method, and high-precision waveform data (e.g., 44.1kHz sampling rate, 16-bit depth) is generated using real-time audio rendering based on a physical modeling synthesis method. Audio effects processing is added (e.g., setting a 2.5-second decay time for the reverb effect and adjusting the frequency response of the equalizer to enhance spatiality), and all tracks are mixed and encoded into a standard audio format (e.g., a multi-track mixed WAV file) to generate complete music data.

[0083] It should be noted that the dynamic multitrack synthesis method is an audio generation method based on timestamp alignment. Its core function is to perform layered rendering and mixing of the macroscopic music structure generated by the high-level manager strategy network and the microscopic music elements generated by the low-level actuator strategy network. By acquiring the volume balance, sound image positioning and effects chain processing of each track in real time, it generates stereo audio data with spatial hierarchy, completing the conversion from symbolic music information to waveform data.

[0084] S4: Based on complete music data and combined with music theory rules, perform multimodal music quality assessment to generate reward signals, and combine historical popular music database to calculate style similarity and market potential index to generate a multi-objective reward function;

[0085] S4.1: Extract acoustic features from complete music data using a multimodal audio feature extraction method, standardize the acoustic features, and generate a set of standardized feature vectors.

[0086] Specifically, acoustic features (such as Mel frequency cepstral coefficients, spectral centroids, and zero-crossing rate features) are extracted from complete music data using multimodal audio feature extraction methods. The acoustic feature values ​​are scaled to a uniform range or Z-scores are standardized to a mean of 0 and a variance of 1 to generate a set of standardized feature vectors. The entire process ensures feature consistency and comparability, providing input for subsequent music quality assessment.

[0087] It should be noted that the multimodal audio feature extraction method is a comprehensive processing method that simultaneously extracts multiple types of acoustic features from complete music data. Specifically, these include time-domain features such as zero-crossing rate features, frequency-domain features, and perceptual features. These features are extracted in parallel from the original audio waveform using digital signal processing algorithms (such as fast Fourier transform and Mel filter bank) to form a multidimensional feature vector, ensuring that the comprehensive attributes of the music signal are covered, and providing a rich and consistent input basis for subsequent standardization and music quality assessment.

[0088] S4.2: Perform multimodal music quality assessment on the standardized feature vector set, and generate a reward signal by combining music theory rules;

[0089] Furthermore, when conducting multimodal music quality assessment on the standardized feature vector set, music attribute parameters of the standardized feature vector set are extracted. These parameters are then combined with harmonic rules from music theory to obtain harmony compliance, rhythmic stability based on rhythmic organization principles, and melodic fluency based on melody writing norms to form a structural assessment dimension. Simultaneously, emotional intensity values ​​and spectral feature distributions from the multimodal features are integrated to form an expressive assessment dimension. The results of the structural and expressive assessment dimensions are then input into a weighted algorithm for fusion to generate a comprehensive reward signal.

[0090] S4.3: Calculate style similarity and market potential index based on reward signals and the feature set of popular songs in the historical hot song database;

[0091] Furthermore, based on the reward signal and combined with the feature set of popular songs in the historical hot song database (such as style tags like pop and rock, play count metrics like millions of plays, and emotional features like the proportion of positive emotions), style similarity is calculated through a similarity calculation algorithm, and a market potential index is calculated using regression analysis nodes. The entire process ensures that style similarity and market potential index reflect real market preferences, providing data support for music optimization.

[0092] The formula for calculating style similarity is:

[0093] ;

[0094] in, Indicating stylistic similarity, Indicates the number of the current musical work 3D eigenvalues This indicates the number of popular songs in the historical hot songs database. 3D eigenvalues This indicates the total number of feature dimensions.

[0095] The formula for calculating the market potential index is:

[0096] ;

[0097] in, Indicating market potential index, This represents the intercept term of the regression analysis node. Indicates the first The regression coefficients of each feature Indicates the first One characteristic variable, This represents the total number of characteristic variables.

[0098] It should be noted that the historical popular songs database is a dedicated dataset constructed by collecting and integrating multi-dimensional features of historically popular musical works. Its generation process includes: collecting metadata, play count metrics, user interaction data, and audio features of popular songs; extracting acoustic features through audio signal processing methods, identifying emotional features of lyrics through natural language processing methods, and obtaining musical structural features through music information retrieval methods; and forming a queryable set of popular song features through data cleaning, feature standardization, and structured storage, providing benchmark data support for the calculation of market potential index.

[0099] S4.4: Integrate market potential index and reward signals to generate a multi-objective reward function.

[0100] Furthermore, the market potential index and reward signals are integrated, and the values ​​are scaled to a uniform dimension through normalization. A multi-objective weighted summation function is used to obtain the comprehensive reward value. The weight allocation is dynamically adjusted in combination with constraints (such as music theory rules to ensure priority of artistry) to generate a multi-objective reward function. This function is used to quantify the multi-dimensional evaluation of music quality and provide a comprehensive and balanced feedback mechanism for subsequent reinforcement learning strategy optimization.

[0101] S5: Based on the multi-objective reward function, the policy networks of the high-level manager and the low-level executor are updated simultaneously through the proximal policy optimization algorithm, and the final complete music data and music creation analysis report are generated.

[0102] S5.1: The multidimensional reward function is analyzed using a near-end policy optimization algorithm to generate standardized reward data;

[0103] Specifically, the multidimensional reward function is analyzed using the proximal policy optimization algorithm (a reinforcement learning algorithm whose core design goal is to ensure the stability and efficiency of policy updates. The proximal policy optimization algorithm limits the step size of each policy update by incorporating a pruning mechanism, preventing training collapse caused by overly aggressive changes in the new policy relative to the old policy. The proximal policy optimization algorithm optimizes a new objective function that includes pruning terms and an objective function, ensuring smooth convergence of the training process while fully utilizing sampled data). The importance sampling method is then employed (importance sampling is used to estimate the expected value of an unknown distribution under a known distribution. In the proximal policy optimization algorithm, the importance sampling method is used to estimate the expected value of an unknown distribution under a known distribution.). Importance sampling is specifically used to obtain the probability ratio between the current policy and the old policy. This probability ratio reflects the likelihood that the new policy will take the same action given the empirical data collected under the old policy. Through importance sampling, the proximal policy optimization algorithm can reuse data collected under the old policy for multiple policy updates. It obtains the probability ratio between the current policy and the old policy (where "current policy" refers to the network parameters being trained for the new policy to be optimized, and "old policy" refers to the network parameters used when empirical data was collected before the update). A pruning mechanism limits the magnitude of the ratio to avoid over-updating. Then, a generalized advantage estimation function is used to obtain the advantage value based on the reward signal, generating standardized reward data.

[0104] S5.2: Based on standardized reward data, obtain the policy network update gradient through the near-end policy optimization algorithm and generate the policy network update gradient vector;

[0105] Specifically, based on standardized reward data, the near-end policy optimization algorithm obtains the policy network, and the gradient ascent algorithm, combined with the objective function optimization, derives the policy network update gradient (e.g., the gradient value is based on the partial derivative of the policy loss function), forming the policy network update gradient vector (e.g., a floating-point vector containing the weight update direction, with dimensions consistent with the network parameters). This sequence of operations ensures the accuracy of gradient derivation and the convergence of policy iteration.

[0106] S5.3: Update gradient vectors based on policy networks to update the parameters of the policy networks of the high-level manager and the low-level executor;

[0107] Furthermore, based on the gradient vector update of the policy network, the Adam optimizer is used in conjunction with the learning rate coefficient to synchronously update the parameters of the high-level manager policy network and the low-level executor policy network. This ensures coordinated optimization of all weights and biases, achieving efficient updating of policy network parameters and thus enhancing the performance and stability of the music generation strategy. The Adam optimizer provides dynamic learning rate adjustment during this process, improving convergence efficiency.

[0108] S5.4: Based on the updated high-level manager and low-level executor policy network, it generates the final complete music data and music creation analysis report through multi-level music generation and intelligent analysis methods.

[0109] Specifically, based on the updated high-level manager policy network and low-level executor policy network, a multi-level music generation process is initiated: the high-level manager policy network processes semantic vector data through the LSTM policy network to generate macroscopic music structures, while the low-level executor policy network simultaneously processes symbolic action sequences through the Transformer decoder to generate microscopic music elements. The macroscopic structure and microscopic elements are integrated through a dynamic multi-track synthesis method, and audio waveforms are generated by combining real-time audio rendering. Intelligent analysis methods perform multimodal quality assessment to generate complete music data and a music creation analysis report.

[0110] This embodiment also provides a computer device applicable to the reinforcement learning-based AI music creation information processing method, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the reinforcement learning-based AI music creation information processing method proposed in the above embodiment.

[0111] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0112] This embodiment also provides a storage medium storing a computer program, which, when executed by a processor, implements the AI ​​music creation information processing method for reinforcement learning as proposed in the above embodiments. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0113] In summary, this invention achieves accurate identification of plot climaxes and emotional turning points by inputting semantic vector data into an LSTM-based policy network, thereby enhancing the emotional expressiveness and narrative quality of the generated music; and through graph traversal and path search algorithms of a dynamic music knowledge graph, it achieves intelligent selection and combination of musical elements, thereby enhancing the narrative fit and professional expressiveness of the music.

[0114] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A reinforcement learning-based AI method for processing music creation information, characterized in that: include, We collect semantic vector data of the plot development trajectory, emotional change curve and multimodal feature fusion of the series, and use LSTM-based policy network to identify the climax and emotional turning point of the plot to obtain the macro music structure. The macroscopic music structure is input into a dynamic music knowledge graph. The optimal element path is searched through graph traversal and path search algorithms. Combined with rhythmic suggestions and orchestration cues, symbolic action sequences are generated. The symbolic action sequence is decoded into a specific note sequence by the Transformer decoder, and the attention mechanism is used to make smooth transitions between notes. The generation parameters are fine-tuned according to the real-time context to generate complete music data. Based on complete music data and combined with music theory rules, multimodal music quality assessment is performed to generate reward signals. Furthermore, style similarity and market potential index are calculated by combining historical popular music databases to generate a multi-objective reward function. Based on the multi-objective reward function, the policy networks of the high-level manager and the low-level executor are updated simultaneously through the proximal policy optimization algorithm, and the final complete music data and music creation analysis report are generated.

2. The AI ​​music composition information processing method based on reinforcement learning as described in claim 1, characterized in that: The method employs an LSTM-based policy network to identify plot climaxes and emotional turning points, thereby obtaining the macroscopic music structure. The specific steps are as follows: Semantic vector data is input into an LSTM-based policy network, and the hidden state and output probability distribution at each time step are obtained through forward propagation of the network to generate an enhanced temporal feature representation. Based on temporal features, a list of plot climaxes and emotional turning points is identified through peak detection and change point detection. By combining the list of plot climaxes and emotional turning points with the time distribution, the boundaries of musical segments are divided and segment types are assigned to generate a macroscopic musical structure.

3. The AI ​​music creation information processing method based on reinforcement learning as described in claim 2, characterized in that: The process involves inputting the macroscopic music structure into a dynamic music knowledge graph, and then searching for the optimal element path using graph traversal and path search algorithms. The specific steps are as follows: Input the macroscopic music structure into the dynamic music knowledge graph, perform multi-element analysis, and generate a set of graph query commands; Based on the set of graph query instructions, the optimal element path is generated by graph traversal and path search, and a set of candidate music element paths is generated.

4. The AI ​​music creation information processing method based on reinforcement learning as described in claim 3, characterized in that: The process of generating symbolic action sequences by combining rhythmic suggestions and orchestration cues involves the following steps. Based on the candidate music element path set, the rhythmic characteristics of music concept nodes are identified through multi-level analysis methods to generate rhythmic suggestions; By combining rhythmic suggestions with an instrument timbre database using a multi-dimensional dynamic mapping method, orchestration prompts are generated. The candidate music element path set, rhythm pattern suggestions, and orchestration cues are integrated to generate a symbolic action sequence.

5. The AI ​​music composition information processing method based on reinforcement learning as described in claim 4, characterized in that: The process involves decoding the symbolic action sequence into a specific note sequence using a Transformer decoder, and then using an attention mechanism to achieve smooth transitions between notes. The specific steps are as follows. The symbolic action sequence is converted into readable sequence data, and the sequence data is decoded into a specific note sequence by the Transformer decoder; Based on the specific note sequence, an attention mechanism is used to optimize the smooth transition between notes, generating an optimized note sequence.

6. The AI ​​music composition information processing method based on reinforcement learning as described in claim 5, characterized in that: The steps for fine-tuning the generated parameters based on real-time context to generate complete music data are as follows. Based on the optimized note sequence, the pitch, duration, and velocity parameters are fine-tuned in real time context to obtain the set of fine-tuned note parameters; Based on the finely tuned set of note parameters, complete music data is generated through dynamic multitrack synthesis and real-time audio rendering.

7. The AI ​​music composition information processing method based on reinforcement learning as described in claim 6, characterized in that: The process of generating reward signals through multimodal music quality assessment based on complete music data and in accordance with music theory rules involves the following specific steps. Acoustic features are extracted from complete music data using a multimodal audio feature extraction method. The acoustic features are then standardized to generate a set of standardized feature vectors. Multimodal music quality assessment is performed on a standardized feature vector set, and a reward signal is generated by combining music theory rules.

8. The AI ​​music composition information processing method based on reinforcement learning as described in claim 7, characterized in that: The process of combining historical popular song databases to calculate style similarity and market potential index to generate a multi-objective reward function involves the following steps: Based on the reward signals, and combined with the feature set of popular songs in the historical hot song database, the style similarity and market potential index are calculated. By integrating market potential indices and reward signals, a multi-objective reward function is generated.

9. The AI ​​music composition information processing method based on reinforcement learning as described in claim 8, characterized in that: The specific steps for updating the policy networks of both the high-level manager and the low-level executor simultaneously using a proximal policy optimization algorithm based on a multi-objective reward function are as follows: The multidimensional reward function is analyzed using a proximal policy optimization algorithm to generate standardized reward data. Based on standardized reward data, the policy network update gradient is obtained through a near-end policy optimization algorithm, and the policy network update gradient vector is generated. The gradient vector is updated based on the policy network, thus updating the parameters of the policy networks of the high-level manager and the low-level executor.

10. The AI ​​music composition information processing method based on reinforcement learning as described in claim 9, characterized in that: The final complete music data and music creation analysis report are generated based on the updated high-level manager and low-level executor strategy network, through multi-level music generation and intelligent analysis methods.

Images