An AI music generation method and system based on BPM rhythm matching

Abstract

This invention discloses an AI music generation method and system based on BPM rhythm matching, belonging to the field of artificial intelligence music generation technology. It involves acquiring a user's music library, constructing a two-dimensional classification matrix based on music category and emotional tone data, and summarizing music by combination. For each sub-music block, the audio signal is framed, and MFCC, chroma, and rhythm feature vectors are extracted. These are then concatenated and combined with the mean and maximum values ​​of multiple frames to obtain a comprehensive feature vector for the sub-music block. When an AI music generation request is received, sub-music blocks from the user's listening history are retrieved, filtered according to BPM requirements, and weighted based on playback frequency to obtain a reference comprehensive feature vector. The comprehensive playback frequency of the secondary classification combinations is counted and queued. Combinations are sequentially taken from the queue, and sub-music blocks that match the BPM are selected. A similarity index is calculated to filter candidate sub-music blocks, and AI music is generated based on a GAN model. This achieves a method for generating AI music based on user preferences and BPM rhythm matching.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence music generation technology, and in particular to an AI music generation method and system based on BPM rhythm matching. Background Technology

[0002] With the rapid development of artificial intelligence technology, significant progress has been made in the field of AI music generation. Traditional music creation mainly relies on the inspiration, professional knowledge and composition skills of human composers. The creation process is time-consuming and limited by the composer's own experience and style. The emergence of AI music generation technology has brought new possibilities to music creation. It uses algorithms such as machine learning and deep learning to learn and analyze a large amount of music data, and can automatically generate musical works with a certain style and expressiveness.

[0003] Currently, AI music generation technology has been applied in many fields, such as game music creation, film and television scores, and advertising music production. However, existing AI music generation methods still have some shortcomings. For example, existing methods may not be comprehensive enough in extracting musical features, only considering some features, such as rhythm or pitch, while ignoring other important musical features. This may limit the AI ​​music generation system's ability to understand and express musical content. Secondly, existing methods may not fully consider users' personalized preferences, such as music genre, emotional tone, and listening habits, which makes it difficult for AI music generation systems to meet the unique preferences of different users. Summary of the Invention

[0004] (a) Technical problems to be solved

[0005] To address the technical problems in the background art, this invention proposes an AI music generation method and system based on BPM rhythm matching. By extracting MFCC features, chroma features, and rhythm features, and performing multi-dimensional feature fusion, candidate sub-music blocks are selected through the calculation of weight coefficients and similarity indices, in combination with the user's music category preferences, emotional tone preferences, and listening record data; thereby solving the technical problems described in the background art.

[0006] (II) Technical Solution

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] An AI music generation method based on BPM rhythm matching includes:

[0009] Obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; divide each piece of music into blocks, and label the generated sub-music blocks with BPM value and two-dimensional classification combination category;

[0010] The audio signal corresponding to each sub-music block is divided into frames, and MFCC features, chroma features, and rhythm features are extracted to obtain the MFCC feature vector, chroma feature vector, and rhythm feature vector of each frame. The three feature vectors are concatenated to obtain the high-dimensional feature vector of each frame. The mean and maximum value of the feature vectors of multiple frames are combined to obtain the comprehensive feature vector of each sub-music block.

[0011] When an AI music generation request is received, the system retrieves sub-music blocks of each song from the user's listening history data, filters sub-music blocks based on the BPM requirement value, calculates the weight coefficient of each corresponding sub-music block based on the number of times each song has been played, and combines them to obtain a reference comprehensive feature vector under the corresponding BPM requirement value; based on the user's listening history data, the system counts and calculates the comprehensive playback frequency of each secondary classification combination, sorts it, and puts it into the matching waiting queue.

[0012] A secondary classification combination is obtained sequentially from the matching waiting queue. All sub-music blocks that meet the BPM requirement under this secondary classification combination are selected. The comprehensive feature vector of each sub-music block is obtained sequentially, and the similarity index with the reference comprehensive feature vector is calculated. Candidate sub-music blocks are selected, and AI music is generated based on the GAN model and combined with all candidate sub-music blocks.

[0013] Specifically, the music library contains n music categories and m emotional tone data. A two-dimensional classification matrix is ​​constructed by combining all the music category data and emotional tone data in the music library, resulting in n×m secondary classification combinations. All music provided by all users in the music library is aggregated, and each piece of music is matched with its corresponding secondary classification combination.

[0014] Specifically, the high-frequency components are increased by using a high-pass filter to perform pre-emphasis; the audio signal corresponding to each sub-musical block is divided into frames; and a window function is applied to each frame of the signal after framing.

[0015] The pre-emphasized and windowed single-frame audio signal is subjected to Fast Fourier Transform to obtain the frequency domain signal of each frame audio signal;

[0016] Calculate the power spectrum for each frame to quantify the spectral energy distribution;

[0017] The spectrum is mapped to a Mel frequency scale, and the spectrum is weighted and averaged through a set of triangular filters, with the logarithm of each value output by the filter bank being taken.

[0018] By performing a discrete cosine transform on the logarithmic Mel spectrum, the Mel frequency cepstral coefficients are obtained, resulting in a 13-dimensional MFCC eigenvector.

[0019] Specifically, the audio signal corresponding to each sub-music block is divided into frames, and the short-time energy of each frame is calculated.

[0020] Based on the difference in energy between two adjacent frames, determine whether each frame is a starting point and determine the intensity of the starting point;

[0021] The beat position of the audio signal is analyzed based on the autocorrelation method, and the beat frequency of each frame is calculated.

[0022] The energy characteristics of each frame in the audio signal, the intensity of the starting point, and the beat probability are combined to form a 3D rhythm feature vector.

[0023] Furthermore, the audio signal corresponding to each sub-music block is framed, the short-time Fourier transform of the audio signal is calculated to obtain the spectrum, the spectrum is divided into 12 sub-bands, and the energy of each sub-band is calculated, which is a 12-dimensional chroma feature vector.

[0024] The 13-dimensional MFCC feature vector, 12-dimensional chroma feature vector, and 3-dimensional rhythm feature vector of each frame in the audio signal are concatenated to obtain a 28-dimensional feature vector for each frame.

[0025] The average value of the 28-dimensional feature vectors of all frames in the audio signal is calculated at the corresponding positions to obtain the first 28-dimensional feature vector. The maximum value at each position of the 28-dimensional feature vectors of all frames in the audio signal is obtained and combined in order to obtain the second 28-dimensional feature vector. The first 28-dimensional feature vector and the second 28-dimensional feature vector are concatenated to obtain the 56-dimensional comprehensive feature vector of the corresponding audio signal, which is the comprehensive feature vector of each sub-musical block.

[0026] Specifically, the system retrieves the music corresponding to the listening records, as well as the music category and emotional tone data for each piece of music; and retrieves several sub-music blocks obtained after dividing each music block.

[0027] Obtain the BPM requirement value from the AI ​​music generation request, select sub-music blocks from the obtained sub-music blocks whose BPM values ​​are the same as the BPM requirement value, and assign a weight coefficient λ to the selected sub-music blocks. i The calculation is as follows: Where i represents the i-th selected sub-music block, c i This represents the number of times the music belonging to the i-th sub-music block has been played, and M represents the total number of sub-music blocks selected.

[0028] Obtain the combined feature vector of all selected sub-musical blocks. And based on the weight λ of each sub-musical block i Calculate the reference comprehensive feature vector of the current BPM requirement value. The expression is:

[0029] Furthermore, based on the music category and emotional tone data of each song in the listening record data, the frequency of music playback for each music category and the frequency of music playback for each emotional tone are counted respectively.

[0030] Add the playback frequency of music corresponding to any music category to the playback frequency of music corresponding to any emotional tone to obtain the comprehensive playback frequency of the corresponding secondary category combination. Similarly, calculate the comprehensive playback frequency of each secondary category combination in turn, sort all secondary category combinations in descending order of comprehensive playback frequency, and put them into the matching waiting queue in order.

[0031] Furthermore, a secondary classification combination is obtained sequentially from the matching waiting queue, and all sub-musical blocks under that secondary classification combination are obtained.

[0032] From all the acquired sub-musical blocks, select those with BPM values ​​that match the required BPM value, and calculate the comprehensive feature vector of each selected sub-musical block based on Euclidean distance. Combined with reference feature vector The similarity index Si j ;

[0033] The combined feature vector of each filtered sub-musical block Combined with reference feature vector The similarity index Si j Sort the data in descending order of similarity; set a similarity index threshold Si0, and select the similarity indices Si in order of secondary classification combination. j Several sub-musical blocks with a similarity index greater than the threshold Si0 are selected as candidate sub-musical blocks.

[0034] Specifically, let's denote the comprehensive feature vector. The form is {x j1 ,x j2 ,…x j56}, referencing the comprehensive feature vector The form is {x1, x2, ... x 56};

[0035] The similarity index Si j The expression for Si is: j =α1*MSi j +α2*CSi j+α3*RSi j , where α1, α2, and α3 represent the weight coefficients of the MFCC feature vector, chroma feature vector, and rhythm feature vector, respectively, and α1+α2+α3=1;

[0036] MSi j Let be the similarity index of MFCC feature vectors, expressed as:

[0037] CSi j The similarity index of chroma feature vectors is expressed as:

[0038] RSi j The similarity index for rhythm feature vectors is expressed as:

[0039] An AI music generation system based on BPM rhythm matching includes:

[0040] The music classification and summarization module is used to obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; it performs block operation on each piece of music, and marks the generated sub-music blocks with BPM value and two-dimensional classification combination category;

[0041] The feature extraction and processing module is used to divide the audio signal corresponding to each sub-music block into frames, and extract MFCC features, chroma features and rhythm features frame by frame to obtain the MFCC feature vector, chroma feature vector and rhythm feature vector of each frame; the three feature vectors are concatenated to obtain the high-dimensional feature vector of each frame; the mean and maximum value of the feature vectors of multiple frames are combined to obtain the comprehensive feature vector of each sub-music block.

[0042] When the AI ​​music matching module receives an AI music generation request, it obtains sub-music blocks of each song in the user's listening record data, filters sub-music blocks based on the BPM requirement value, calculates the weight coefficient of each corresponding sub-music block based on the number of times each song is played, and combines them to obtain a reference comprehensive feature vector under the corresponding BPM requirement value; based on the user's listening record data, it counts and calculates the comprehensive playback frequency of each secondary classification combination, sorts it, and puts it into the matching waiting queue.

[0043] The AI ​​music generation module is used to sequentially obtain a secondary classification combination from the matching waiting queue, filter out all sub-music blocks that meet the BPM requirement under the secondary classification combination, sequentially obtain the comprehensive feature vector of the sub-music blocks and calculate the similarity index with the reference comprehensive feature vector, filter the candidate sub-music blocks, and generate AI music based on the GAN model and all candidate sub-music blocks.

[0044] (III) Beneficial Effects

[0045] This invention provides an AI music generation method and system based on BPM rhythm matching, which has the following beneficial effects:

[0046] 1. By acquiring the user's music library, a two-dimensional classification matrix is ​​constructed based on music category and emotional tone data, and the music is summarized. Then, the music is divided into blocks and labeled with BPM value and category combination. It can accurately integrate user music preferences and provide a rich and orderly music material library for subsequent AI music generation based on user preferences and rhythm matching, making the generated AI music more in line with the user's personalized needs.

[0047] 2. By dividing each sub-musical block into frames, extracting MFCC, chroma, and rhythm features frame by frame, and concatenating them to obtain a high-dimensional feature vector, and then combining the mean and maximum values ​​of multiple frames to obtain a comprehensive feature vector; comprehensively and meticulously mining the feature information of the music signal, making the generated AI music more in line with the target requirements at the feature level, providing accurate and effective feature basis for subsequent music similarity calculation and AI music generation, and helping to generate high-quality AI music that matches the reference features;

[0048] 3. After receiving the AI ​​music generation request, the system filters sub-music blocks based on the user's listening history and BPM requirements, calculates weight coefficients to obtain a reference comprehensive feature vector, counts the playback frequency of secondary classification combinations and sorts them into a matching queue, filters candidate sub-music blocks and generates AI music based on the GAN model; taking into account the user's recent listening habits and rhythm requirements, the system generates AI music by reasonably filtering and matching music blocks and using the GAN model, so that the generated AI music not only meets the user's preferences but also satisfies specific rhythm requirements. Attached Figure Description

[0049] Figure 1 A schematic diagram illustrating the steps of an AI music generation method based on BPM rhythm matching provided by the present invention;

[0050] Figure 2 This is a schematic diagram of the structure of an AI music generation system based on BPM rhythm matching provided by the present invention. Detailed Implementation

[0051] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] refer to Figure 1This invention provides an AI music generation method based on BPM rhythm matching, comprising:

[0053] Step 1: Obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; divide each piece of music into blocks, and label the generated sub-music blocks with BPM value and two-dimensional classification combination category;

[0054] Step one includes the following steps:

[0055] Step 101: Assign an ID to each user and obtain the music library provided by each user; the music library stores all the music that the user wants to integrate. This invention needs to combine the user's preferences and compare the music libraries provided by several users to create AI music that matches the user's preferences.

[0056] The music library stores music category data and emotional tone data for each piece of music. The music category data refers to information classifying and labeling musical works according to style, genre, or type, such as rock, pop, classical, jazz, etc., used to describe the overall style and characteristics of the music; there are n types of music category data, specifically using A... a This represents the data for the a-th music category;

[0057] The emotional tone data refers to information describing the emotional tendency or atmosphere expressed by a musical work, such as joy, sadness, excitement, or tranquility, used to convey the emotional experience the music evokes in the listener; there are m types of emotional tone data, specifically using B... b This represents the data for the b-th sentiment tone.

[0058] Each piece of music simultaneously belongs to a music category and a mood tone.

[0059] Step 102: Compile all music libraries provided by users, and further categorize and summarize all music according to different music categories and emotional tones, specifically as follows:

[0060] By combining all music category data and emotional tone data and constructing a two-dimensional classification matrix, n×m groups of secondary classification combinations are obtained;

[0061] It aggregates all music in the music libraries provided by all users, matches each piece of music with the corresponding secondary category combination, and assigns each piece of music the same tag as the user who provided it. The same piece of music can have tags from multiple users, meaning that the same piece of music exists in the music libraries provided by multiple users at the same time.

[0062] Step 103: Sequentially obtain each piece of music under each secondary classification combination, divide each piece of music into blocks, specifically into 30-second sub-music blocks. If the duration of the last sub-music block after dividing a piece of music is less than 30 seconds, then extend the sub-music block by extending it 30 seconds forward from the end of the piece of music, and take the sub-music block corresponding to these 30 seconds as the last sub-music block of the corresponding piece of music.

[0063] Each sub-musical block is labeled with its corresponding music name, and the BPM value of each sub-musical block is obtained using audio analysis software; based on the music described in each sub-musical block, each sub-musical block is matched one-to-one with a secondary classification combination.

[0064] When using it, refer to steps 101 to 103:

[0065] By acquiring users' music libraries, constructing a two-dimensional classification matrix based on music category and emotional tone data, and summarizing the music, the music is then divided into blocks and labeled with BPM values ​​and category combinations. This allows for the accurate integration of users' music preferences, providing a rich and orderly music library for subsequent AI music generation based on user preferences and rhythm matching, making the generated AI music more in line with users' personalized needs.

[0066] Step 2: Divide the audio signal corresponding to each sub-music block into frames, and extract MFCC features, chroma features, and rhythm features frame by frame to obtain the MFCC feature vector, chroma feature vector, and rhythm feature vector of each frame; concatenate the three feature vectors to obtain the high-dimensional feature vector of each frame; combine the mean and maximum value of the feature vectors of multiple frames to obtain the comprehensive feature vector of each sub-music block.

[0067] Step two includes the following steps:

[0068] Step 201: Extract music features from the audio signal corresponding to each sub-music block. The music features specifically include Mel frequency cepstral coefficient features, i.e., MFCC features, chroma features, and rhythm features.

[0069] Step 202: MFCC features are used to capture the spectral characteristics of audio signals. Based on the non-linear perception of frequency by the human ear (Mel scale), feature extraction of audio signals is performed. The specific steps are as follows:

[0070] Step 2021: Increase the energy of the high-frequency components by using a high-pass filter to achieve pre-emphasis. This enhances the high-frequency components in the audio signal, compensates for possible attenuation of the high-frequency signal during transmission, and makes the spectrum flatter. The expression is: s l '(k)=s l (k)-β·s l (k-1), where sl '(k) represents the pre-emphasized signal, s l (k) represents the k-th sampling point of the l-th frame, and β represents the pre-emphasis coefficient;

[0071] Step 2022: Perform a frame division operation on the audio signal corresponding to each sub-music block. Specifically, divide the audio signal into several short frames with a frame length of 30ms, and make 50% overlap between adjacent frames.

[0072] Step 2023: Apply a window function to each frame of the signal to reduce spectral leakage and make the frequency domain analysis more accurate; the specific window function is the Hamming window function, and its expression is: Where N represents the window length, i.e., the number of sampling points, and w(k) represents the window function value; multiplying the window function by each frame of signal point by point, the expression is:

[0073] Step 2024: Perform a Fast Fourier Transform on the pre-emphasized and windowed single-frame audio signal to obtain the frequency domain signal of each frame, expressed as: Among them, S l (o) represents the frequency of the l-th frame, and I represents the imaginary unit;

[0074] Step 2025: Calculate the power spectrum of each frame to quantize the spectral energy distribution, expressed as: Among them, P l (o) represents the power spectrum of the l-th frame;

[0075] Step 2026: Map the spectrum to a Mel frequency scale and perform a weighted average of the spectrum using a set of triangular filters; the relationship between the Mel frequency and the ordinary frequency f is:

[0076] Take the logarithm of each value output by the filter bank to simulate the human ear's perception of sound intensity;

[0077] The logarithmic Mel spectrum is subjected to discrete cosine transform to obtain Mel frequency cepstral coefficients. Since the lower-order coefficients contain the main spectral information and the higher-order coefficients are often related to noise, the first 12 to 13 coefficients are usually retained. Here, the first 13 coefficients are taken to obtain a 13-dimensional MFCC eigenvector.

[0078] Step 203: Chromaticity features reflect the color characteristics of an audio signal that do not contain luminance information, and are used to describe the hue and saturation of the audio signal;

[0079] Specifically, the audio signal is divided into several short frames with a frame length of 30ms. The spectrum is obtained by calculating the short-time Fourier transform of the audio signal. Then, the spectrum is divided into 12 sub-bands. The energy of each sub-band is calculated and normalized to eliminate the influence of volume changes. Finally, the 12-dimensional chroma feature vector represents the energy distribution of the audio signal in different sub-bands.

[0080] Step 204: Rhythmic features are time-related features in audio signals, specifically including energy features, starting point intensity, and beat probability. They describe information such as the beat intensity and rhythmic variations of the audio signal.

[0081] Step 2041: Specifically, the audio signal is divided into several short frames with a frame length of 30ms, and the short-time energy E(l) of each frame is calculated, with the expression as follows: Where s(l+k) represents the value of the audio signal at the l+kth sampling point;

[0082] Step 2042: Detect whether each frame is a starting point (a sudden change point in energy or spectrum). First, calculate the energy difference D(l) between two adjacent frames, expressed as: D(l) = E(l) - E(l-1). Set a difference threshold. If the energy difference D(l) between two adjacent frames exceeds the difference threshold, then mark the l-th frame as a starting point. For each frame, if it is a starting point, then the starting point intensity S(l) = D(l); otherwise, the starting point intensity S(l) = 0.

[0083] Step 2043: Estimate the beat position of the audio signal based on the autocorrelation method, and calculate the autocorrelation function R(τ), which is expressed as: Where τ represents the delay, that is, the time offset when the signal is compared with itself; R(τ) is calculated sequentially starting from τ=1 to obtain the autocorrelation sequence; the first peak is found in the autocorrelation sequence, and the delay corresponding to the peak is taken as the beat period T, that is, the beat position of the audio signal is estimated as K·T, and K∈{1,2,...};

[0084] Calculate the beat probability P(l) for each frame, expressed as follows: Where l is the index of the current frame, l near It is the index of the nearest beat to the current frame, and σ represents the standard deviation, which is one-quarter of the beat interval.

[0085] Step 2044: Combine the energy features, starting point intensity, and beat probability of each frame in the audio signal to form a 3D rhythm feature vector;

[0086] Step 205: Concatenate the 13-dimensional MFCC feature vector, 12-dimensional chroma feature vector, and 3-dimensional rhythm feature vector of each frame in the audio signal to obtain a 28-dimensional feature vector for each frame.

[0087] The average value of the 28-dimensional feature vectors of all frames in the audio signal is calculated at the corresponding positions to obtain the first 28-dimensional feature vector. The maximum value at each position of the 28-dimensional feature vectors of all frames in the audio signal is obtained and combined in order to obtain the second 28-dimensional feature vector. The first 28-dimensional feature vector and the second 28-dimensional feature vector are concatenated to obtain the 56-dimensional comprehensive feature vector of the corresponding audio signal.

[0088] Based on the above steps, the comprehensive feature vector of each sub-musical block is analyzed and obtained.

[0089] When using this method, refer to steps 201 to 205:

[0090] By segmenting each sub-musical block into frames, extracting MFCC, chroma, and rhythm features frame by frame, and concatenating them to obtain a high-dimensional feature vector, and then combining the mean and maximum values ​​of multiple frames to obtain a comprehensive feature vector, the feature information of the music signal is comprehensively and meticulously mined, making the generated AI music more in line with the target requirements at the feature level. This provides accurate and effective feature basis for subsequent music similarity calculation and AI music generation, and helps to generate high-quality AI music that matches the reference features.

[0091] Step 3: Upon receiving an AI music generation request, obtain sub-music blocks for each song in the user's listening history data. Filter sub-music blocks based on the BPM requirement value and calculate the weight coefficient of each corresponding sub-music block based on the number of times each song is played. Combine these to obtain a reference comprehensive feature vector under the corresponding BPM requirement value. Based on the user's listening history data, count and calculate the comprehensive playback frequency of each secondary classification combination, sort it, and place it in the matching waiting queue. Sequentially obtain a secondary classification combination from the matching waiting queue, filter out all sub-music blocks under that secondary classification combination that meet the BPM requirement value, sequentially obtain the comprehensive feature vector of each sub-music block, and calculate the similarity index with the reference comprehensive feature vector to filter candidate sub-music blocks. Generate AI music based on the GAN model and by combining all candidate sub-music blocks.

[0092] Step three includes the following steps:

[0093] Step 301: When a user sends an AI music generation request, obtain the user's listening history data. The music corresponding to each user's listening history data comes from the music library they provide.

[0094] Obtain all music played in the user's listening history data and the corresponding number of plays within the last few days;

[0095] Step 302: Obtain the music corresponding to the listening record data, as well as the music category and emotional tone data of each music; obtain several sub-music blocks obtained after dividing each music block;

[0096] Obtain the BPM requirement value from the AI ​​music generation request, select sub-music blocks from the obtained sub-music blocks whose BPM values ​​are the same as the BPM requirement value, and assign a weight coefficient λ to the selected sub-music blocks. i The calculation is as follows: Where i represents the i-th selected sub-music block, c i The number of times the music to which the i-th sub-music block belongs is played is represented, and M represents the total number of selected sub-music blocks; the weight coefficient represents the reference index of each sub-music block. When generating AI music, it is necessary to consider the user's recent listening habits in order to intelligently generate AI music that the user is most likely to be interested in.

[0097] Step 303: Obtain the comprehensive feature vector of all selected sub-musical blocks. And based on the weight λ of each sub-musical block i Calculate the reference comprehensive feature vector of the current BPM requirement value. The expression is:

[0098] Step 304: Based on the music category and emotional tone data of each song in the listening record data, count the frequency of music playback for each music category and the frequency of music playback for each emotional tone. Specifically, the sum of the number of times all songs under each music category in the listening record data are played is taken as the frequency of music playback for the corresponding music category. The same method is used to count the frequency of music playback for each emotional tone.

[0099] Add the music playback frequency corresponding to any music category to the music playback frequency corresponding to any emotional tone to obtain the comprehensive playback frequency of the corresponding secondary category combination. Similarly, calculate the comprehensive playback frequency of each secondary category combination in turn, sort all the secondary category combinations in descending order of comprehensive playback frequency, and put them into the matching waiting queue in order.

[0100] Step 305: Sequentially obtain a secondary classification combination from the matching waiting queue, and obtain all sub-music blocks under the secondary classification combination. The sub-music blocks obtained here include several sub-music blocks from the music libraries provided by all users. Combining the music libraries of multiple users helps to generate more reasonable AI music.

[0101] From all the acquired sub-musical blocks, select those with BPM values ​​that match the required BPM value, and calculate the comprehensive feature vector of each selected sub-musical block based on Euclidean distance. Combined with reference feature vector The similarity index Si j Let the comprehensive eigenvector be denoted as . The form is {x j1 ,x j2 ,…x j56}, referencing the comprehensive feature vector The form is {x1, x2, ... x 56}, where the elements of terms 1 to 13 and terms 29 to 41 are elements of the MFCC feature vector, then the similarity index MSi of the MFCC feature vector is... j for

[0102] Items 14 through 25, and items 42 through 53, are elements of the chroma feature vector. Therefore, the similarity index CSi of the chroma feature vector is... j for;

[0103] Items 26 to 28 and 54 to 56 are elements of the rhythm feature vector, and the similarity index RSi of the rhythm feature vector is... j for;

[0104] The similarity index MSi combining MFCC feature vectors j The similarity index CSi of chromaticity feature vectors j and the similarity index RSi of rhythm feature vectors j The comprehensive feature vector is obtained. Combined with reference feature vector The similarity index Si j The expression is: Si j =α1*MSi j +α2*CSi j +α3*RSi j , where α1, α2, and α3 represent the weight coefficients of the MFCC feature vector, chroma feature vector, and rhythm feature vector, respectively. The specific values ​​are determined by the user input, and α1+α2+α3=1;

[0105] Step 306: Combine the feature vectors of each filtered sub-musical block. Combined with reference feature vector The similarity index Si jSort the data in descending order of similarity; set a similarity index threshold Si0, and select the similarity indices Si in order of secondary classification combination. j Several sub-music blocks with a similarity index threshold Si0 are selected as candidate sub-music blocks; the number of selected sub-music blocks is determined by the user, and the length of a sub-music block is 30 seconds; if the number of candidate sub-music blocks selected in the current secondary classification combination does not reach the number of sub-music blocks selected by the user, the next secondary classification combination in the matching waiting queue will be selected in turn to select candidate sub-music blocks;

[0106] The comprehensive feature vector corresponding to each candidate sub-music block is used as the input condition of the GAN model; a generator is constructed to receive a random noise vector and output the generated music fragment; a discriminator is constructed to receive the music fragment and output the probability value that the fragment is real; the parameters of the generator and discriminator are randomly initialized, and adversarial training is performed, specifically: the generator generates a music fragment and passes it to the discriminator, the discriminator judges the authenticity of the music fragment and calculates the loss, and updates the parameters of the generator and discriminator according to the loss; the above adversarial training process is repeated until the model converges; the matched music features are used as the input condition of the generator, and the generator generates new music fragments according to the input features; the music fragments generated by each candidate sub-music block are spliced ​​together to obtain the intelligently generated AI music.

[0107] When using this method, refer to steps 301 to 306:

[0108] After receiving the AI ​​music generation request, the system filters sub-music blocks based on the user's listening history and BPM requirements, calculates weight coefficients to obtain a reference comprehensive feature vector, and sorts the playback frequency of secondary classification combinations into a matching queue. Candidate sub-music blocks are then filtered and AI music is generated based on the GAN model. Taking into account the user's recent listening habits and rhythm requirements, the system generates AI music by reasonably filtering and matching music blocks and using the GAN model, ensuring that the generated AI music meets both user preferences and specific rhythm requirements.

[0109] refer to Figure 2 The present invention also provides an AI music generation system based on BPM rhythm matching, comprising:

[0110] The music classification and summarization module is used to obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; it performs block operation on each piece of music, and marks the generated sub-music blocks with BPM value and two-dimensional classification combination category;

[0111] The feature extraction and processing module is used to divide the audio signal corresponding to each sub-music block into frames, and extract MFCC features, chroma features and rhythm features frame by frame to obtain the MFCC feature vector, chroma feature vector and rhythm feature vector of each frame; the three feature vectors are concatenated to obtain the high-dimensional feature vector of each frame; the mean and maximum value of the feature vectors of multiple frames are combined to obtain the comprehensive feature vector of each sub-music block.

[0112] When the AI ​​music matching module receives an AI music generation request, it obtains sub-music blocks of each song in the user's listening record data, filters sub-music blocks based on the BPM requirement value, calculates the weight coefficient of each corresponding sub-music block based on the number of times each song is played, and combines them to obtain a reference comprehensive feature vector under the corresponding BPM requirement value; based on the user's listening record data, it counts and calculates the comprehensive playback frequency of each secondary classification combination, sorts it, and puts it into the matching waiting queue.

[0113] The AI ​​music generation module is used to sequentially obtain a secondary classification combination from the matching waiting queue, filter out all sub-music blocks that meet the BPM requirement under the secondary classification combination, sequentially obtain the comprehensive feature vector of the sub-music blocks and calculate the similarity index with the reference comprehensive feature vector, filter the candidate sub-music blocks, and generate AI music based on the GAN model and all candidate sub-music blocks.

[0114] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present invention are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer storage medium or transmitted through a computer storage medium.

[0115] Computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. Computer storage media can be any available medium that a computer can access, or a data storage device such as a server or data center that integrates one or more available media. Available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state drives (SSDs)).

[0116] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this invention should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An AI music generation method based on BPM rhythm matching, characterized in that: Includes the following steps: Obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; divide each piece of music into blocks, and label the generated sub-music blocks with BPM value and two-dimensional classification combination category; The music library contains a total of Music category data and Based on the emotional tone data, a two-dimensional classification matrix is ​​constructed by combining all music category data and emotional tone data in the music library, resulting in... Grouping and secondary classification; The audio signal corresponding to each sub-music block is divided into frames, and MFCC features, chroma features, and rhythm features are extracted to obtain the MFCC feature vector, chroma feature vector, and rhythm feature vector of each frame. The three feature vectors are concatenated to obtain the high-dimensional feature vector of each frame. The mean and maximum value of the feature vectors of multiple frames are combined to obtain the comprehensive feature vector of each sub-music block. Upon receiving an AI music generation request, the system retrieves sub-music blocks from each song in the user's listening history. These sub-music blocks are then filtered based on the required BPM value, and a weight coefficient for each corresponding sub-music block is calculated based on the number of plays for each song. These weight coefficients are then combined to obtain a reference comprehensive feature vector for the corresponding BPM requirement. Specifically: Obtain the music corresponding to the listening record data, as well as the music category and emotional tone data of each music; obtain several sub-music blocks obtained after dividing each music block; Obtain the BPM requirement value from the AI ​​music generation request, select sub-music blocks from the obtained sub-music blocks whose BPM values ​​are the same as the BPM requirement value, and assign weight coefficients to the selected sub-music blocks. The calculation is as follows: ,in, Indicates the selected first Individual music blocks, Indicates the first The number of times the music to which each sub-music block belongs is played. This indicates the total number of selected sub-musical blocks; Obtain the combined feature vector of all selected sub-musical blocks. And based on the weight of each sub-musical block Calculate the reference comprehensive feature vector of the current BPM requirement value. The expression is: ; Based on users' listening history data, the overall playback frequency of each secondary category combination is statistically calculated and sorted before being placed into a matching waiting queue. Specifically: Based on the music category and emotional tone data of each song in the listening record data, the frequency of music playback for each music category and the frequency of music playback for each emotional tone are counted respectively. Add the music playback frequency corresponding to any music category to the music playback frequency corresponding to any emotional tone to obtain the comprehensive playback frequency of the corresponding secondary category combination. Similarly, calculate the comprehensive playback frequency of each secondary category combination in turn, sort all the secondary category combinations in descending order of comprehensive playback frequency, and put them into the matching waiting queue in order. A secondary classification combination is obtained sequentially from the matching waiting queue. All sub-music blocks that meet the BPM requirement under this secondary classification combination are selected, including several sub-music blocks from all user-provided music libraries. The comprehensive feature vectors of the sub-music blocks are obtained sequentially, and the similarity index with the reference comprehensive feature vector is calculated respectively. Candidate sub-music blocks are selected, and AI music is generated based on the GAN model and combined with all candidate sub-music blocks.

2. The AI ​​music generation method based on BPM rhythm matching as described in claim 1, characterized in that: It aggregates all music from the music libraries provided by all users and matches each piece of music with its corresponding secondary category combination.

3. The AI ​​music generation method based on BPM rhythm matching as described in claim 1, characterized in that: The high-frequency components are enhanced by using a high-pass filter to perform pre-emphasis; the audio signal corresponding to each sub-musical block is divided into frames; and a window function is applied to each frame of the signal after framing to perform windowing. The pre-emphasized and windowed single-frame audio signal is subjected to Fast Fourier Transform to obtain the frequency domain signal of each frame audio signal; Calculate the power spectrum for each frame to quantify the spectral energy distribution; The spectrum is mapped to a Mel frequency scale, and the spectrum is weighted and averaged through a set of triangular filters, with the logarithm of each value output by the filter bank being taken. By performing a discrete cosine transform on the logarithmic Mel spectrum, the Mel frequency cepstral coefficients are obtained, resulting in a 13-dimensional MFCC eigenvector.

4. The AI ​​music generation method based on BPM rhythm matching as described in claim 3, characterized in that: Perform frame segmentation on the audio signal corresponding to each sub-music block and calculate the short-time energy of each frame; Determine whether each frame is a starting point based on the energy difference between two adjacent frames, and determine the intensity of the starting point; The beat position of the audio signal is analyzed based on the autocorrelation method, and the beat frequency of each frame is calculated. The energy characteristics of each frame in the audio signal, the intensity of the starting point, and the beat probability are combined to form a 3D rhythm feature vector.

5. The AI ​​music generation method based on BPM rhythm matching as described in claim 4, characterized in that: The audio signal corresponding to each sub-music block is framed, and the short-time Fourier transform of the audio signal is calculated to obtain the spectrum. The spectrum is divided into 12 sub-bands, and the energy of each sub-band is calculated, which is a 12-dimensional chroma feature vector. The 13-dimensional MFCC feature vector, 12-dimensional chroma feature vector, and 3-dimensional rhythm feature vector of each frame in the audio signal are concatenated to obtain a 28-dimensional feature vector for each frame. The average value of the 28-dimensional feature vectors of all frames in the audio signal is calculated at the corresponding positions to obtain the first 28-dimensional feature vector. The maximum value at each position of the 28-dimensional feature vectors of all frames in the audio signal is obtained and combined in order to obtain the second 28-dimensional feature vector. The first 28-dimensional feature vector and the second 28-dimensional feature vector are concatenated to obtain the 56-dimensional comprehensive feature vector of the corresponding audio signal, which is the comprehensive feature vector of each sub-musical block.

6. The AI ​​music generation method based on BPM rhythm matching as described in claim 1, characterized in that: Sequentially retrieve one secondary classification combination from the matching waiting queue, and then retrieve all sub-musical blocks under that secondary classification combination; From all the acquired sub-musical blocks, select those with BPM values ​​that match the required BPM value, and calculate the comprehensive feature vector of each selected sub-musical block based on Euclidean distance. Combined with reference feature vector Similarity index ; The combined feature vector of each filtered sub-musical block Combined with reference feature vector Similarity index Sort in descending order; set a similarity index threshold. The similarity indices are selected sequentially according to the order of the secondary classification combination. Greater than the similarity index threshold Several sub-musical blocks are selected as candidate sub-musical blocks.

7. The AI ​​music generation method based on BPM rhythm matching as described in claim 6, characterized in that: Let the comprehensive eigenvector be... The form is Reference comprehensive feature vector The form is ; The similarity index The expression is: ,in, , , Let represent the weight coefficients of the MFCC feature vector, chroma feature vector, and rhythm feature vector, respectively. ; Let be the similarity index of MFCC feature vectors, expressed as: ; The similarity index of chroma feature vectors is expressed as: ; The similarity index for rhythm feature vectors is expressed as: .

8. An AI music generation system based on BPM rhythm matching, used to implement the method described in any one of claims 1 to 7, characterized in that, include: The music classification and summarization module is used to obtain the music library provided by the user, construct a two-dimensional classification matrix based on music category data and emotional tone data, and summarize the music in the music library according to the two-dimensional classification combination; it performs block operation on each piece of music, and marks the generated sub-music blocks with BPM value and two-dimensional classification combination category; The feature extraction and processing module is used to divide the audio signal corresponding to each sub-music block into frames, and extract MFCC features, chroma features and rhythm features frame by frame to obtain the MFCC feature vector, chroma feature vector and rhythm feature vector of each frame; the three feature vectors are concatenated to obtain the high-dimensional feature vector of each frame; the mean and maximum value of the feature vectors of multiple frames are combined to obtain the comprehensive feature vector of each sub-music block. When the AI ​​music matching module receives an AI music generation request, it obtains sub-music blocks of each song in the user's listening record data, filters sub-music blocks based on the BPM requirement value, calculates the weight coefficient of each corresponding sub-music block based on the number of times each song is played, and combines them to obtain a reference comprehensive feature vector under the corresponding BPM requirement value; based on the user's listening record data, it counts and calculates the comprehensive playback frequency of each secondary classification combination, sorts it, and puts it into the matching waiting queue. The AI ​​music generation module is used to sequentially obtain a secondary classification combination from the matching waiting queue, filter out all sub-music blocks that meet the BPM requirement under the secondary classification combination, sequentially obtain the comprehensive feature vector of the sub-music blocks and calculate the similarity index with the reference comprehensive feature vector, filter the candidate sub-music blocks, and generate AI music based on the GAN model and all candidate sub-music blocks.

Images