An automatic and vocal generation method and system based on multi-dimensional feature matching

Abstract

This invention relates to the field of audio signal processing technology, and in particular to an automatic harmony generation method and system based on multi-dimensional feature matching. By introducing a context-aware mechanism and streaming iterative optimization, it achieves a leap from static feature matching to dynamic music semantic understanding and generation. It not only extracts basic features such as pitch and rhythm in real time, but also integrates high-level contextual information such as harmonic tension trends, musical form positions, and emotional energy, so that the generated harmony can fit the overall development and emotional flow of the music. Through the collaboration of the harmony decision fusion model and the streaming iterative optimizer, the system can continuously make dynamic adjustments and smooth corrections based on real-time input and user feedback during the performance. This significantly improves the system's real-time interactivity, melody following, and creative expressiveness while ensuring the rationality of the harmonic music, and can effectively meet the intelligent harmony assistance needs in highly dynamic scenarios such as improvisation and interactive music generation.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, and in particular to an automatic harmony generation method and system based on multidimensional feature matching. Background Technology

[0002] Harmony is a core element in musical composition and performance. Through the vertical combination of different notes, it endows music with rich color, emotional tension, and structural logic. Traditional harmony design relies on the composer's profound music theory knowledge, experience, and inspiration, making it a high-barrier and time-consuming process. With the application of computer technology and artificial intelligence in the music field, automatic harmony generation has gradually become an important research direction. It aims to assist musicians in quickly generating harmonic progressions that conform to musical logic, reducing the difficulty of composition, improving efficiency, and showing broad application potential in areas such as improvisation, real-time accompaniment, and music education.

[0003] However, existing automatic harmony generation methods focus on static matching of basic features such as pitch and rhythm, lacking real-time perception and integration of musical context information, making it difficult to meet the creation and performance needs in highly interactive and real-time scenarios. Summary of the Invention

[0004] The purpose of this invention is to provide an automatic harmony generation method and system based on multi-dimensional feature matching, which solves the technical problem in the prior art of lacking real-time perception and fusion of musical context information, making it difficult to meet the creation and performance needs in highly interactive and real-time scenarios.

[0005] To achieve the above objectives, this invention provides an automatic harmony generation method based on multidimensional feature matching, comprising: Receives input audio signal streams from musical instruments or human voices; The audio signal stream is analyzed in real time to extract real-time basic music features, including pitch, duration, and rhythm pattern. Combining current and historical musical characteristics, real-time contextual features, including the trend of harmonic tension changes, the position of musical form, and the direction of emotional energy, are calculated through a context-aware mechanism. Based on a pre-set music theory rule base, the real-time basic music features are initially screened to generate a primary harmony candidate set. The primary harmony candidate set, real-time basic music features, and real-time contextual features are input together into a pre-trained harmony decision fusion model. The harmony decision fusion model integrates a decision tree classifier and a neural network to evaluate the matching degree between candidate harmonies and multi-dimensional features, and outputs a preliminary harmony scheme. The initial harmony scheme is input into the streaming iterative optimizer. Based on the real-time audio characteristics of subsequent inputs and real-time user feedback, the iterative optimizer smoothly corrects and dynamically adjusts the harmony within a time window to generate the final harmony track. The final harmony track is time-aligned and mixed with the original audio signal for output. The alignment process uses a real-time alignment module based on a dynamic time warping algorithm to ensure precise rhythmic synchronization between the harmony and the main melody.

[0006] The specific methods for receiving input audio signal streams from musical instruments or human voices include: When the transport layer receives a transmission task, the management layer requests the key generator to generate an encryption key and a decryption key. Then, the encryption layer encrypts the original data based on the encryption key. After that, the transport layer transmits the ciphertext to the designated module and uses the decryption key to decrypt and obtain the original audio signal.

[0007] The automatic harmony generation method based on multidimensional feature matching integrates a permissioned blockchain network, stores key process data on the blockchain through smart contracts, and uses this to build a federated learning data pool. It then uses association rules and clustering analysis algorithms to mine optimization patterns and drive the continuous collaborative evolution of the harmony generation model.

[0008] The context-aware mechanism analyzes historical feature sequences through a sliding time window, and its working principle includes: Construct a tonality / mode state tracker based on a hidden Markov model to infer the most likely tonality center and stability in real time; By analyzing the harmonic tension values ​​of consecutive measures using recurrent neural network units, we can predict their short-term variation trends. Based on pre-labeled musical form templates or repeated passages discovered through unsupervised learning, determine the relative position of the current musical phrase within the overall musical structure.

[0009] The training and operation methods of the pre-trained harmony decision fusion model include: During the training phase, a large number of music clips labeled with high-quality harmonies are used as samples. The multidimensional features of the samples are used as input, and the chord progression sequence is used as the output target for supervised learning. During the working phase, the decision tree branch first performs a rapid screening based on tonality and strong beat rules. The neural network submodule then performs a fine scoring on the selected candidate harmonies. The scoring comprehensively considers the interval consonance with the main melody, the smoothness of the connection with the preceding and following chords, and whether it conforms to the current emotional color.

[0010] The streaming iterative optimizer executes the following algorithm: A fixed-length first-in-first-out buffer is established to continuously receive the latest preliminary harmony scheme and corresponding audio characteristics; Within the buffer, a smoothing function based on weighted least squares is used to fine-tune the harmony in the time domain, ensuring natural transitions between adjacent chords; The system detects user feedback signals through the interactive interface in real time. If a negation or modification instruction for the harmony of a section is detected, the area is immediately marked as a high-weight area, and local re-optimization is initiated. At the same time, this interaction is used as a reinforcement learning sample to update the decision fusion model.

[0011] The neural network submodule in the harmony decision fusion model is a multi-head attention mechanism network, and its operation includes: Each chord in the primary harmony candidate set is encoded into a vector, and attention is computed between it and a "query" vector representing real-time basic features and contextual features. The correlation scores between candidate chords and melody pitch, rhythm intensity, and emotional characteristics are calculated in parallel by multiple attention heads. Finally, these scores are summed to obtain the final matching score for each candidate chord.

[0012] The present invention also provides an automatic harmony generation system based on multidimensional feature matching, for performing the automatic harmony generation method based on multidimensional feature matching as described above. include: Audio stream input and real-time feature extraction module; Context-aware computing module; Music theory rule base and primary candidate generation module; The Harmony Decision Fusion Model module contains a pre-trained ensemble learning model; Streaming Iterative Optimizer Module; User interaction and personalized learning module; Harmony timbre mapping and final synthesis output module.

[0013] This invention discloses an automatic harmony generation method and system based on multi-dimensional feature matching. This solution achieves a leap from static feature matching to dynamic musical semantic understanding and generation by introducing a context-aware mechanism and streaming iterative optimization. Specifically, the system not only extracts basic features such as pitch and rhythm in real time, but also integrates high-level contextual information such as harmonic tension trends, formal positions, and emotional energy, enabling the generated harmony to fit the overall development and emotional flow of the music. Furthermore, through the collaboration of a harmonic decision fusion model and a streaming iterative optimizer, the system can continuously make dynamic adjustments and smooth corrections based on real-time input and user feedback during performance. This significantly improves the system's real-time interactivity, melody responsiveness, and creative expressiveness while ensuring the rationality of the harmonic music, effectively meeting the intelligent harmony assistance needs in highly dynamic scenarios such as improvisation and interactive music generation. Attached Figure Description

[0014] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below.

[0015] Figure 1 This is a flowchart of the automatic harmony generation method based on multidimensional feature matching of the present invention. Detailed Implementation

[0016] The embodiments of the present invention are described in detail below. Examples of the embodiments are shown in the accompanying drawings. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, but should not be construed as limiting the present invention.

[0017] Please see Figure 1 , Figure 1 This is a flowchart of the automatic harmony generation method based on multidimensional feature matching of the present invention.

[0018] This invention provides an automatic harmony generation method and system based on multidimensional feature matching, comprising: S1. Receive input audio signal streams from musical instruments or human voices; In this specific embodiment, the specific methods for receiving input audio signal streams from musical instruments or human voices include: When the transport layer receives a transmission task, the management layer requests the key generator to generate an encryption key and a decryption key. Then, the encryption layer encrypts the original data based on the encryption key. After that, the transport layer transmits the ciphertext to the designated module and uses the decryption key to decrypt and obtain the original audio signal.

[0019] S2. Perform real-time analysis on the audio signal stream to extract real-time basic music features including pitch, duration, and rhythm pattern; S3. Combining current and historical musical characteristics, calculate real-time contextual features, including the trend of harmonic tension changes, the position of the musical form, and the direction of emotional energy, through a context-aware mechanism. In this specific implementation, the context-aware mechanism analyzes historical feature sequences through a sliding time window, and its operation includes: Construct a tonality / mode state tracker based on a hidden Markov model to infer the most likely tonality center and stability in real time; By analyzing the harmonic tension values ​​of consecutive measures using recurrent neural network units, we can predict their short-term variation trends. Based on pre-labeled musical form templates or repeated passages discovered through unsupervised learning, determine the relative position of the current musical phrase within the overall musical structure.

[0020] S4. Based on the preset music theory rule base, perform initial screening of real-time basic music features to generate a primary harmony candidate set; S5. Input the primary harmony candidate set, real-time basic music features, and real-time context features into a pre-trained harmony decision fusion model. The harmony decision fusion model integrates a decision tree classifier and a neural network to evaluate the matching degree between candidate harmonies and multi-dimensional features, and outputs a preliminary harmony scheme. For this specific implementation method, the training and operation of the pre-trained harmony decision fusion model includes: During the training phase, a large number of music clips labeled with high-quality harmonies are used as samples. The multidimensional features of the samples are used as input, and the chord progression sequence is used as the output target for supervised learning. During the working phase, the decision tree branch first performs a rapid screening based on tonality and strong beat rules. The neural network submodule then performs a fine scoring on the selected candidate harmonies. The scoring comprehensively considers the interval consonance with the main melody, the smoothness of the connection with the preceding and following chords, and whether it conforms to the current emotional color.

[0021] The neural network submodule in the harmony decision fusion model is a multi-head attention mechanism network, and its operation includes: Each chord in the primary harmony candidate set is encoded into a vector, and attention is computed between it and a "query" vector representing real-time basic features and contextual features. The correlation scores between candidate chords and melody pitch, rhythm intensity, and emotional characteristics are calculated in parallel by multiple attention heads. Finally, these scores are summed to obtain the final matching score for each candidate chord.

[0022] S6. Input the preliminary harmony scheme into the streaming iterative optimizer. The iterative optimizer performs smooth correction and dynamic adjustment of the harmony within the time window based on the real-time audio characteristics of subsequent inputs and real-time user feedback, generating the final harmony track. For this specific implementation, the streaming iterative optimizer executes the following algorithm: A fixed-length first-in-first-out buffer is established to continuously receive the latest preliminary harmony scheme and corresponding audio characteristics; Within the buffer, a smoothing function based on weighted least squares is used to fine-tune the harmony in the time domain, ensuring natural transitions between adjacent chords; The system detects user feedback signals through the interactive interface in real time. If a negation or modification instruction for the harmony of a section is detected, the area is immediately marked as a high-weight area, and local re-optimization is initiated. At the same time, this interaction is used as a reinforcement learning sample to update the decision fusion model.

[0023] S7. The final harmony track is time-aligned and mixed with the original audio signal for output. The alignment process uses a real-time alignment module based on dynamic time warping algorithm to ensure that the harmony and the main melody are precisely synchronized in rhythm.

[0024] The automatic harmony generation method based on multidimensional feature matching integrates a permissioned blockchain network, stores key process data on the chain through smart contracts, and uses this to build a federated learning data pool. It then uses association rules and clustering analysis algorithms to mine optimization patterns and drive the continuous collaborative evolution of the harmony generation model.

[0025] The automatic harmony generation method based on multidimensional feature matching also includes a personalized learning step: The system creates independent feature weight configuration files for different users or music styles; Record users' adoption, modification, or rejection of generated harmonies, and store them in association with the multidimensional feature context at the time of generation to form a personal preference dataset; By periodically using this personal preference dataset, user-specific parameters in the harmony decision fusion model are fine-tuned to achieve personalized adaptation of harmony style.

[0026] After generating the chord progression sequence, the system intelligently matches or synthesizes a timbre with the corresponding emotional color (such as warm, bright, or dark) for each chord from a timbre material library, based on real-time emotional color characteristics and user preset styles. The final output harmony track is a composite audio stream that combines pitch sequences with dynamic timbre changes.

[0027] The present invention also provides an automatic harmony generation system based on multidimensional feature matching, for performing the automatic harmony generation method based on multidimensional feature matching as described above. include: Audio stream input and real-time feature extraction module; Context-aware computing module; Music theory rule base and primary candidate generation module; The Harmony Decision Fusion Model module contains a pre-trained ensemble learning model; Streaming Iterative Optimizer Module; User interaction and personalized learning module; Harmony timbre mapping and final synthesis output module.

[0028] Using the automatic harmony generation method and system based on multi-dimensional feature matching in this embodiment, this solution achieves a leap from static feature matching to dynamic music semantic understanding and generation by introducing a context-aware mechanism and streaming iterative optimization. Specifically, the system not only extracts basic features such as pitch and rhythm in real time, but also integrates high-level contextual information such as harmonic tension trends, formal positions, and emotional energy, enabling the generated harmony to fit the overall development and emotional flow of the music. On this basis, through the collaboration of the harmony decision fusion model and the streaming iterative optimizer, the system can continuously make dynamic adjustments and smooth corrections based on real-time input and user feedback during the performance. This significantly improves the system's real-time interactivity, melody following, and creative expressiveness while ensuring the rationality of the harmonic music, effectively meeting the intelligent harmony assistance needs in highly dynamic scenarios such as improvisation and interactive music generation.

[0029] The above-disclosed embodiments are merely one or more preferred embodiments of this application and should not be construed as limiting the scope of this application. Those skilled in the art can understand that all or part of the processes for implementing the above embodiments and equivalent changes made in accordance with the claims of this application still fall within the scope of this application.

Claims

1. An automatic harmony generation method based on multidimensional feature matching, characterized in that, include: Receives input audio signal streams from musical instruments or human voices; The audio signal stream is analyzed in real time to extract real-time basic music features, including pitch, duration, and rhythm pattern. Combining current and historical musical characteristics, real-time contextual features, including the trend of harmonic tension changes, the position of musical form, and the direction of emotional energy, are calculated through a context-aware mechanism. Based on a pre-set music theory rule base, the real-time basic music features are initially screened to generate a primary harmony candidate set. The primary harmony candidate set, real-time basic music features, and real-time contextual features are input together into a pre-trained harmony decision fusion model. The harmony decision fusion model integrates a decision tree classifier and a neural network to evaluate the matching degree between candidate harmonies and multi-dimensional features, and outputs a preliminary harmony scheme. The initial harmony scheme is input into the streaming iterative optimizer. Based on the real-time audio characteristics of subsequent inputs and real-time user feedback, the iterative optimizer smoothly corrects and dynamically adjusts the harmony within a time window to generate the final harmony track. The final harmony track is time-aligned and mixed with the original audio signal for output. The alignment process uses a real-time alignment module based on a dynamic time warping algorithm to ensure precise rhythmic synchronization between the harmony and the main melody.

2. The automatic harmony generation method based on multidimensional feature matching as described in claim 1, characterized in that, Specific methods for receiving input audio signal streams from musical instruments or human voices include: When the transport layer receives a transmission task, the management layer requests the key generator to generate an encryption key and a decryption key. Then, the encryption layer encrypts the original data based on the encryption key. After that, the transport layer transmits the ciphertext to the designated module and uses the decryption key to decrypt and obtain the original audio signal.

3. The automatic harmony generation method based on multidimensional feature matching as described in claim 2, characterized in that, The automatic harmony generation method based on multidimensional feature matching integrates a permissioned blockchain network, stores key process data on the chain through smart contracts, and uses this to build a federated learning data pool. It then uses association rules and clustering analysis algorithms to mine optimization patterns and drive the continuous collaborative evolution of the harmony generation model.

4. The automatic harmony generation method based on multidimensional feature matching as described in claim 3, characterized in that, Context-aware mechanisms analyze historical feature sequences through a sliding time window, and their operation includes: Construct a tonality / mode state tracker based on a hidden Markov model to infer the most likely tonality center and stability in real time; By analyzing the harmonic tension values ​​of consecutive measures using recurrent neural network units, we can predict their short-term variation trends. Based on pre-labeled musical form templates or repeated passages discovered through unsupervised learning, determine the relative position of the current musical phrase within the overall musical structure.

5. The automatic harmony generation method based on multidimensional feature matching as described in claim 4, characterized in that, The training and operation methods of pre-trained harmony decision fusion models include: During the training phase, a large number of music clips labeled with high-quality harmonies are used as samples. The multidimensional features of the samples are used as input, and the chord progression sequence is used as the output target for supervised learning. During the working phase, the decision tree branch first performs a rapid screening based on tonality and strong beat rules. The neural network submodule then performs a fine scoring on the selected candidate harmonies. The scoring comprehensively considers the interval consonance with the main melody, the smoothness of the connection with the preceding and following chords, and whether it conforms to the current emotional color.

6. The automatic harmony generation method based on multidimensional feature matching as described in claim 5, characterized in that, The streaming iterative optimizer executes the following algorithms: A fixed-length first-in-first-out buffer is established to continuously receive the latest preliminary harmony scheme and corresponding audio characteristics; Within the buffer, a smoothing function based on weighted least squares is used to fine-tune the harmony in the time domain, ensuring natural transitions between adjacent chords; The system detects user feedback signals through the interactive interface in real time. If a negation or modification instruction for the harmony of a section is detected, the area is immediately marked as a high-weight area, and local re-optimization is initiated. At the same time, this interaction is used as a reinforcement learning sample to update the decision fusion model.

7. The automatic harmony generation method based on multidimensional feature matching as described in claim 6, characterized in that, The neural network submodule in the harmony decision fusion model is a multi-head attention mechanism network, and its operation includes: Each chord in the primary harmony candidate set is encoded as a vector, and attention is computed between it and a "query" vector representing real-time basic features and contextual features. The correlation scores between candidate chords and melody pitch, rhythm intensity, and emotional characteristics are calculated in parallel by multiple attention heads. Finally, these scores are summed to obtain the final matching score for each candidate chord.

8. An automatic harmony generation system based on multidimensional feature matching, used to execute the automatic harmony generation method based on multidimensional feature matching as described in claim 7, characterized in that, include: Audio stream input and real-time feature extraction module; Context-aware computing module; Music theory rule base and primary candidate generation module; The Harmony Decision Fusion Model module contains a pre-trained ensemble learning model; Streaming Iterative Optimizer Module; User interaction and personalized learning module; Harmony timbre mapping and final synthesis output module.

Images