Music playing method and device, electronic equipment and readable storage medium

Abstract

The embodiment of the invention discloses a music playing method and device, electronic equipment and a readable storage medium. According to the embodiment of the invention, after the advancing speed and the heart rate of the user are obtained, the rhythm parameter of the to-be-played music is determined according to the advancing speed and the heart rate of the user, the candidate music files are determined from the preset music file library according to the rhythm parameter, and then the to-be-played target music file is selected from the candidate music files. And playing the target music file. In the embodiment of the invention, each music file in the preset music file library is generated based on the pre-trained music generation model, and the music generation model is obtained by training according to the sample audio file, the musical instrument digital interface file of the sample audio file, the attribute information and the note sequence, so that the richness of music can be increased; and the music playing flexibility can be improved, so that the user experience is improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and more specifically to a music playback method, apparatus, electronic device, and readable storage medium. Background Technology

[0002] In daily life, people can choose from various modes of transportation, such as bicycles, cars, and buses. To enhance the enjoyment of their journeys, some users choose to play music during their travels. However, existing music playback methods cannot be specifically tailored to this, resulting in a poor user experience. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a music playback method, apparatus, electronic device, and readable storage medium to increase the richness of music and improve the flexibility of music playback, thereby enhancing the user experience.

[0004] In a first aspect, embodiments of the present invention provide a music playback method, the method comprising:

[0005] Obtain the target user's walking speed and heart rate;

[0006] The target rhythm parameters are determined based on the travel speed and heart rate.

[0007] Candidate music files are determined from a preset music file library based on the target rhythm parameters. Each preset music file in the preset music file library is obtained based on a pre-trained music generation model. The pre-trained music generation model is trained based on a training sample set. The training sample set includes multiple sample audio files, instrument digital interface files of each sample audio file, attribute information, and note sequences. The attribute information includes the rhythm parameters of the sample audio files.

[0008] Select the target music file from the candidate music files;

[0009] Play the target music file.

[0010] Optionally, determining the target rhythm parameters based on the walking speed and heart rate includes:

[0011] The target rhythm parameter is determined based on the pre-set mapping relationship between walking speed, heart rate, and rhythm parameters, and the walking speed and heart rate.

[0012] Optionally, playing the target music file includes:

[0013] In response to the presence of a non-target music file in playback, the playback mode of the non-target music file is adjusted based on a pre-set music adjustment strategy to match the target music file. The music adjustment strategy includes at least one of adjusting the rhythm parameter of the non-target music file, adjusting the volume of the non-target music file, and adjusting the pitch of the non-target music file.

[0014] In response to the playback method matching the target music file, playback of the non-target music file is stopped, and the target music file is played.

[0015] Optionally, the music generation model includes a feature embedding module, an encoding module, a feature fusion module, a conditional layer, and a conditional decoding module. The training sample set also includes the timbre tags of the sample audio files. The music generation model is trained in the following manner:

[0016] Each of the sample audio files is designated as a specific audio file;

[0017] The input to the feature embedding module is determined based on the specified audio file, and the timbre embedding features of the specified audio file are obtained.

[0018] The input to the encoding module is determined based on the instrument digital interface file, and the hidden state sequence of the specified audio file is obtained;

[0019] The input to the conditional layer is determined based on the attribute information of the specified audio file, and the sample conditional embedding features of the specified audio file are obtained.

[0020] The input to the feature fusion module is determined based on the timbre embedding features and the hidden state sequence, and the fusion features of the specified audio file are obtained.

[0021] The fusion features and the sample conditional embedding features are used as inputs to the conditional decoding module, and the music generation model is trained using the note sequence and the timbre label as training targets until the loss function of the music generation model converges.

[0022] Optionally, determining the input of the feature embedding module based on the specified audio file and obtaining the timbre embedding features of the specified audio file includes:

[0023] Obtain the spectrogram of the specified audio file;

[0024] Convert the spectrogram into the frequency cepstral coefficient features of the specified audio file;

[0025] The timbre embedding feature is obtained by using the frequency cepstral coefficient feature as the input of the feature embedding module.

[0026] Optionally, the music generation model includes a conditional layer and a conditional decoding module;

[0027] The preset music file is generated in the following way:

[0028] Obtain music generation conditions, which are determined based on the target user's preference information;

[0029] The input to the conditional layer is determined based on the music generation conditions, and the conditional embedding features of the preset music file are obtained.

[0030] Using the conditional embedding features as input to the conditional decoding module, the target note sequence of the preset music file is obtained;

[0031] The target note sequence is converted into the preset music file.

[0032] Optionally, converting the target note sequence into the preset music file includes:

[0033] The target note sequence is optimized based on a pre-set optimization algorithm to obtain the optimized target note sequence;

[0034] The optimized target note sequence is converted into the preset music file.

[0035] Optionally, determining the target rhythm parameters based on the walking speed and heart rate includes:

[0036] In response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, the target rhythm parameter is adjusted from a first rhythm parameter to a second rhythm parameter to update the target music file, wherein the first rhythm parameter is greater than the second rhythm parameter.

[0037] Optionally, the method further includes:

[0038] In response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, a warning is issued to the target user to prompt the target user to reduce the travel speed.

[0039] Secondly, embodiments of the present invention provide a music playback device, the device comprising:

[0040] The data acquisition unit is used to acquire the target user's walking speed and heart rate;

[0041] The parameter acquisition unit is used to determine the target rhythm parameters based on the travel speed and heart rate.

[0042] A matching unit is used to determine candidate music files from a preset music file library according to the target rhythm parameters. Each preset music file in the preset music file library is obtained based on a pre-trained music generation model. The pre-trained music generation model is trained based on a training sample set. The training sample set includes multiple sample audio files, instrument digital interface files of each sample audio file, attribute information, and note sequences. The attribute information includes the rhythm parameters of the sample audio files.

[0043] The selection unit is used to select a target music file from the candidate music files;

[0044] The playback unit is used to play the target music file.

[0045] Thirdly, embodiments of the present invention provide a computer program product, the computer program product including a computer program / instruction, which, when executed by a processor, implements the method described in any one of the first aspects.

[0046] Fourthly, embodiments of the present invention provide an electronic device, including a memory and a processor, wherein the memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method described in any one of the first aspects.

[0047] Fifthly, embodiments of the present invention provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method described in any one of the first aspects.

[0048] In this embodiment of the invention, after acquiring the user's walking speed and heart rate, the rhythm parameters of the music to be played are determined based on these parameters. Candidate music files are then selected from a preset music file library based on these rhythm parameters, and finally, the target music file to be played is chosen from the candidate files. In this embodiment, each music file in the preset music file library is generated based on a pre-trained music generation model. This music generation model is trained using sample audio files, instrument digital interface files of the sample audio files, attribute information, and note sequences. Therefore, it increases the richness of the music and improves the flexibility of music playback, thereby enhancing the user experience. Attached Figure Description

[0049] The above and other objects, features and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0050] Figure 1 This is a flowchart of a music playback method according to an embodiment of the present invention;

[0051] Figure 2 This is a flowchart of the method for training a music generation model in an embodiment of the present invention;

[0052] Figure 3 This is a data flow diagram of the music generation model in the training phase of an embodiment of the present invention;

[0053] Figure 4 This is a flowchart illustrating the method for generating a preset music file in an embodiment of the present invention;

[0054] Figure 5 This is a data flow diagram of the music generation model in the application stage of this invention embodiment;

[0055] Figure 6 This is a schematic diagram of a music playback device according to an embodiment of the present invention;

[0056] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0057] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the substance of the present application, well-known methods, processes, flows, elements, and circuits are not described in detail.

[0058] Furthermore, those skilled in the art should understand that the accompanying drawings provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0059] Unless the context explicitly requires it, words such as "including" or "contains" throughout the application should be interpreted as including rather than exclusive or exhaustive; that is, meaning "including but not limited to".

[0060] In the description of this application, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0061] The solutions described in this specification and embodiments, if involving the processing of personal information, will be processed only on the premise of having a legal basis (such as obtaining the consent of the personal information subject, or being necessary for the performance of a contract), and will only be processed within the scope stipulated or agreed upon. A user's refusal to process personal information beyond what is necessary for basic functions will not affect the user's use of basic functions.

[0062] In daily life, users can choose from various modes of transportation, such as cycling, driving, or taking public transport, to meet their travel needs. Taking cycling as an example, it's a healthy and natural form of exercise that enriches people's daily lives while improving their physical fitness. Therefore, more and more people are choosing bicycles as their daily mode of transportation for commuting and exercise. To increase the enjoyment of cycling, some users choose to play music while riding. However, existing music playback methods typically only allow for random or sequential playback of music files from a playlist, or switching music based on user input, without the ability to make specific adjustments. This can lead to a poor user experience or increase the potential for safety hazards, threatening the user's cycling safety.

[0063] In this embodiment, a mobile terminal is used as the execution subject, and the application scenario is a user playing music while cycling. However, those skilled in the art will readily understand that the method of this embodiment is equally applicable when the execution subject is another device, such as a server or an in-vehicle computer, and when the application scenario is another, such as a user playing music while riding in a vehicle or driving.

[0064] Figure 1 This is a flowchart of a music playback method according to an embodiment of the present invention. Figure 1 As shown, the method in this embodiment includes the following steps:

[0065] Step S101: Obtain the target user's walking speed and heart rate.

[0066] In practical applications, wearable devices worn by users, such as smartwatches, can collect the user's heart rate during movement. Therefore, in this step, the target user's mobile terminal (hereinafter referred to as the terminal) can receive the target user's heart rate from the wearable device in a polling manner via a short-range communication protocol, and obtain the target user's cycling speed based on built-in sensors, such as angular velocity sensors and accelerometers, or based on positioning systems, such as GPS (Global Positioning System) and BeiDou Navigation Satellite System.

[0067] Step S102: Determine the target rhythm parameters based on walking speed and heart rate.

[0068] To better match the cycling rhythm of the target user, in this step, the terminal can determine the rhythm parameters of the target music file to be played, i.e., the target rhythm parameters, based on the target user's riding speed and heart rate. In this embodiment, the rhythm parameters, including the target rhythm parameters, can be beats per minute (BPM), grava, largo, lento, larghetto, andante, etc.

[0069] Optionally, the terminal can determine the target rhythm parameter based on a predetermined mapping relationship between travel speed, heart rate, and rhythm parameters, and the target user's cycling speed and heart rate. Depending on the actual settings, the mapping relationship between travel speed, heart rate, and rhythm parameters can be a mapping relationship between a range of travel speed values, a range of heart rate values, and a range of rhythm parameter values, or it can be a mapping relationship between travel speed values, heart rate values, and rhythm parameter values. This embodiment does not impose any limitations.

[0070] For example, when cycling at a speed of 0-15 km / h and a heart rate of 100-120 beats / minute, the BPM range can be 80-100; when cycling at a speed of 15-25 km / h and a heart rate of 120-140 beats / minute, the BPM range can be 100-120; and when cycling at a speed of 25-35 km / h and a heart rate of 140-160 beats / minute, the BPM range can be 120-140.

[0071] Step S103: Determine candidate music files from the preset music file library based on the target rhythm parameters.

[0072] After determining the target rhythm parameters, the terminal can select a preset music file that matches the target rhythm parameters from the preset music file library as a candidate music file. In this step, the terminal can either send a music retrieval request to the server to retrieve candidate music files from the preset music file library in the database, or it can select candidate music files from the local preset music library.

[0073] Depending on the actual playback method of the music file, the preset music file can be a MIDI (Musical Instrument Digital Interface) file or an audio file; this embodiment does not impose any restrictions.

[0074] In this embodiment, each preset music file in the preset music library is a music file obtained based on a pre-trained music generation model. The training sample set used to train the music generation model includes multiple sample audio files, MIDI files of each sample audio file, attribute information, and note sequences, wherein the note sequences and timbre labels of each sample audio file are the training targets of the music generation model.

[0075] The music generation model in this embodiment can be a multi-task learning model. The music generation model mainly includes a conditional layer and a decoding module. The conditional layer receives user-specified or system-preset music generation conditions and processes these conditions into a representation that can be processed by the music generation model. The decoding module is the backbone network of the music generation model, used to decode the representation obtained by the feature embedding module into a new music output. Depending on the actual settings, the decoding module can be a Generative Adversarial Network (GAN), a Recurrent Neural Network (RNN), a Convolutional Neural Network (CNN), a Variational Autoencoder (VAE), a Transformer model, etc. This embodiment does not impose any limitations.

[0076] Simultaneously, music generation models can be trained using teacher forcing. Teacher forcing is a technique for training neural network generative models, primarily applied to sequence-to-sequence (seq2seq) models. Its basic idea is to use notes from real note sequences as input to the decoding module in the next time step during training, rather than using notes generated by the decoding module itself. In this way, the music generation model can more easily learn how real note sequences are generated, thus improving the training effect, effectively reducing the training cost, and increasing the accuracy of the music generation model.

[0077] Depending on the specific setup, the structure of the music generation model can differ between the training and application phases. In one optional implementation, the music generation model may include an encoding module, a feature embedding module, and a feature fusion module during the training phase; and during the application phase, it may include a conditional layer and a decoding module.

[0078] The training process of the music generation model is described below. The training process of the music generation model can be executed by the server. Figure 2 This is a flowchart of the method for training a music generation model in an embodiment of the present invention. For example... Figure 2As shown, in this embodiment, the training process of the music generation model may include the following steps:

[0079] Step S201: Each sample audio file is identified as a designated audio file.

[0080] In this embodiment, the same processing can be performed on each sample audio file. Therefore, in this step, each sample audio file can be identified as a designated audio file.

[0081] Step S202: Determine the input of the feature embedding module based on the specified audio file, and obtain the timbre embedding features of the specified audio file.

[0082] To enhance the music generation model's ability to process the timbre of music files, in this step, the server can obtain the spectrogram of a specified audio file and convert it into frequency cepstral coefficient (FCC) features. These CFC features are then used as input to the feature embedding module to obtain the timbre embedding features of the specified audio file. Since the CFC features are a feature matrix, the feature embedding module used to process them can be a neural network module, such as a CNN or RNN, to better handle them.

[0083] Specifically, a specified audio file can be converted into a spectrogram using a Short-Time Fourier Transform (STFT) or Short-Term Fourier Transform, and the spectrogram can be converted into frequency cepstral coefficient features using a Mel-Frequency Cepstral Coefficients (MFCC) transform.

[0084] Step S203: Determine the input of the encoding module based on the instrument digital interface file, and obtain the hidden state sequence of the specified audio file.

[0085] In this step, the server can use a MIDI file of a specified audio file as input to the encoding module and obtain the hidden state sequence of the specified audio file. The hidden state sequence is the internal state used by the neural network to store and transmit information when processing the input sequence, i.e., the MIDI file. It is updated at each time step, capturing information and contextual relationships in the input sequence, and using the captured information for calculations at the current time step, thereby enabling it to process and understand the complex structures and patterns in the sequence data.

[0086] Step S204: Determine the input of the conditional layer based on the attribute information of the specified audio file, and obtain the sample conditional embedding features of the specified audio file.

[0087] The conditional layer can receive and encode music generation conditions. Therefore, in this step, the server can use the attribute information of a specified audio file as the music generation condition to obtain the sample conditional embedding features of the specified audio file, thereby guiding the training process of the music generation model. Depending on the actual settings, other information may also be included in the music generation conditions; this embodiment does not impose any limitations.

[0088] In this embodiment, the attribute information of the specified audio file may include the rhythm parameters of the specified audio file, as well as style information, structural information, etc., of the specified audio file. Depending on the actual settings, other information may also be included, such as rhythm complexity (e.g., simple, medium, complex, etc.), which is not limited in this embodiment.

[0089] The style information can further include musical style information and emotional information. Musical style information can include pop, folk, rock, ballad, classical, jazz, etc., while emotional information can include cheerful, calm, exciting, sad, etc. The structural information can include intro, chorus, interlude, coda, etc.

[0090] Taking the music generation conditions including the bpm, style information, and structural information of a specified audio file as an example, after the server inputs the music generation conditions into the condition layer, the condition layer can convert the style information of the specified audio file into a first code based on a preset dictionary, and convert the structural information of the specified audio file into a second code based on the preset dictionary. Then, the first code, the second code, and the bpm are converted into conditional embedding features of the specified audio file. Depending on actual needs, the first and second codes can be one-hot or multi-hot encodings; this embodiment does not impose any restrictions.

[0091] It is easy to understand that in this embodiment, steps S202-S204 can be executed simultaneously or sequentially, and there is no restriction here.

[0092] Step S205: Determine the input of the feature fusion module based on the timbre embedding features and the hidden state sequence, and obtain the fusion features of the specified audio file.

[0093] In this step, the server can use the timbre embedding features and hidden state sequence of the specified audio file as input to the feature fusion module to obtain the fusion features of the specified audio file.

[0094] In this embodiment, the feature fusion module can be an attention module. The attention module employs an attention mechanism, which generates more accurate feature representations by selecting key regions in the embedded features and hidden state sequences, and allows the music generation model to focus on different timbre feature hidden state sequences at different time steps.

[0095] Step S206: The fused features and sample conditional embedding features are used as inputs to the conditional decoding module, and the music generation model is trained with note sequences and timbre labels as training targets until the loss function of the music generation model converges.

[0096] In this step, the server can use the fusion features and sample conditional embedding features of the specified audio file as input to the conditional decoding module, and train the music generation model using the note sequence and timbre labels of the specified audio file as training targets until the loss function of the music generation model converges. The music generation model is a multi-task learning model; therefore, its loss function can be a fusion loss function, which can be determined based on the loss function between the true and predicted values ​​of the note sequence and the loss function between the true and predicted values ​​of the timbre labels.

[0097] The timbre tag for a specified audio file can be a tag composed of the timbres of a piano, organ, guitar, brass instrument, reed instrument, percussion instrument, etc. Depending on the actual instrument used to play the specified music file, the timbre tag for the specified music file can be either one-hot encoded or multi-hot encoded; this embodiment does not impose any restrictions.

[0098] For example, if the actual instruments used in audio file 1 are a piano and a violin, the server can set the value of the element corresponding to the piano and violin in the music tag to 1, and set the value of the element corresponding to the other instruments to 0.

[0099] Figure 3 This is a data flow diagram of the music generation model during the training phase in an embodiment of the present invention. For example... Figure 3 As shown, the feature embedding module 31, encoding module 32, conditional layer 33, feature fusion module 34, and conditional decoding module 35 are the main structures of the music generation model.

[0100] The server can convert the sample audio file, i.e., audio 301, into a spectrogram 302, and then convert the spectrogram 302 into MFCC features 303 as input to the feature embedding module 31 to obtain the timbre embedding features 304 of audio 301. The server can also use the MIDI file 305 of audio 301 as input to the encoding module 32 to obtain the hidden state sequence 306 of audio 301. Finally, the server can use the bpm 307, structural information 308, and style information 309 of audio 301 as input to the conditional layer 33 to obtain the sample conditional embedding features 310 of audio 301.

[0101] Furthermore, the server can use the timbre embedding feature 304 and the hidden state sequence 306 of audio 301 as input to the feature fusion module 34 to obtain the fusion feature 311 of audio 301. Then, the sample conditional embedding feature 310 and the fusion feature 311 of audio 301 are used as input to the conditional decoding module 35, and the timbre generation model is trained using the note sequence 312 and timbre label 313 of audio 301 as training targets.

[0102] After training the music generation model, in the application phase, preset music from a pre-defined music library can be generated based on the model. The application process of the music generation model can also be executed by the server. Figure 4 This is a flowchart illustrating the method for generating a preset music file in an embodiment of the present invention. For example... Figure 4 As shown, in this embodiment, the preset music file can be generated through the following steps:

[0103] Step S401: Obtain the music generation conditions.

[0104] In this step, the server can obtain the music generation conditions for the preset music file. To ensure that the generated music better suits the target user's preferences, the music generation conditions can be determined based on the target user's preference information in this step.

[0105] Optionally, the target user's preference information can be determined based on the target user's music playback history. If classical music is the most frequent genre among the music files played by the target user in the music playback history, the server can determine that the target user's preference information is classical music; if the bpm of the music files played by the target user is the most frequent bpm, the server can determine that the target user's preference information is a bpm between 80 and 100.

[0106] Optionally, the target user's preferences can also be set by the target user. The server can obtain music playback-related settings from the target user's settings information as the target user's preferences.

[0107] Step S402: Determine the input of the conditional layer based on the music generation conditions, and obtain the conditional embedding features of the preset music file.

[0108] After obtaining the music generation conditions, the server can use the music generation conditions as input to the condition layer to obtain the conditional embedding features of the preset music file.

[0109] Step S403: Using the conditional embedding features as input to the conditional decoding module, obtain the target note sequence of the preset music file.

[0110] In this step, the server can use the conditional embedding features of the preset music file as input to the conditional decoding module to obtain the target note sequence of the preset music file. The target note sequence is actually a sequence composed of the probability distribution of notes generated at each time step. Therefore, in order to increase the diversity of the preset music file, the server can also adjust the target note sequence through nucleus sampling in this step.

[0111] Specifically, kernel sampling sorts all candidate notes in the probability distribution from highest to lowest probability, and calculates the cumulative probability sum starting from the candidate note with the highest probability. The smallest set of candidate notes whose cumulative probability sum exceeds a preset probability threshold p is the "core" in kernel sampling. This preset probability threshold p typically ranges from 0.9 to 0.95. After determining the "core," the probabilities of each candidate note in the "core" are normalized, and the sum of the probabilities of each candidate note after normalization is 1. The next note is then randomly sampled from the renormalized probability distribution. During the generation of the note sequence, the value of the preset probability threshold p can be dynamically adjusted based on preset musical structure and style information.

[0112] Step S404: Convert the target note sequence into a preset music file.

[0113] In this step, the server can convert the target note sequence into a corresponding preset music file, such as an audio file or a MIDI file, according to the playback method of the music file. This embodiment does not impose any restrictions.

[0114] To improve the adaptability and layering of the preset music file, the server can optionally optimize the generated target note sequence based on a pre-set optimization algorithm, obtain the optimized target note sequence, and convert the optimized target note sequence into a preset music file. Examples of some optimization algorithms are given below.

[0115] Specifically, the server can analyze the rhythmic characteristics of the target note sequence using techniques such as Fast Fourier Transform (FFT) and autocorrelation analysis, and identify the beat positions and bpm of the target note sequence. Then, it aligns the bpm of the target note sequence to the nearest beat grid. For each note in the target note sequence, the server can calculate the deviation of the note's beat position from the ideal time point and adjust the beat position accordingly; alternatively, the server can identify the main rhythmic patterns in the target note sequence and adjust the duration of these rhythmic patterns or each note based on the bpm in the music generation conditions.

[0116] The server can also use time-scaling algorithms, such as WaveformSimilarity Over Lap Add (WSOLA) and phase vocoders, to ensure that the pitch of each note remains constant when adjusting the target note sequence. The server can also align percussion elements to a bpm grid to adjust the beat position of the percussion instruments and balance and adjust the interludes, dynamics, and pitch of the target note sequence. The server can also identify cyclic or repetitive sections in the target note sequence and ensure that the duration of these sections is consistent. The server can also adjust the wave blocking of each note in the target note sequence, including attack, decay, sustain, and release, to adapt to new timeframes. The server can also add or remove embellishing notes in the target note sequence while maintaining the original rhythmic complexity and characteristics of the target note sequence.

[0117] Figure 5 This is a data flow diagram of the music generation model in the application phase of an embodiment of the present invention. For example... Figure 5 As shown, the conditional layer 33 and the conditional decoding module 35 are the main structures of the music generation model.

[0118] The server can input the bpm 501, structural information 502, and style information 503 of the preset music file 506 into the conditional layer 33 as music generation conditions to obtain the conditional embedding features 504 of the preset music file 506. Then, the server uses the conditional embedding features 504 of the preset music file 506 as input to the conditional decoding module 35 to obtain the target note sequence 505 of the preset music file 506. Furthermore, the server can input the target note sequence 505 of the preset music file 506 into the optimization module 50 and convert the optimized target note sequence 505 into the preset music file 506.

[0119] Step S104: Select the target music file from the candidate music files.

[0120] In this step, the terminal can select the target music file from the acquired candidate music files. Optionally, the server can determine the target music file according to various existing methods, such as randomly selecting a candidate music file as the target music file; or, determining the score of each candidate music file based on its rating, cumulative number of plays, cumulative playback duration, etc., and determining the candidate music file with the highest score as the target music file; or, sorting according to a preset rule, such as sorting the music files alphabetically by their first letters, and determining the candidate music file ranked first as the target music file; or, selecting a candidate music file that matches the target user's preference information as the target music file, which is not limited in this embodiment.

[0121] Step S105: Play the target music file.

[0122] After identifying the target music file, in this step, if there is no music file currently playing, the terminal can directly play the target music file; if there is a non-target music file currently playing, the terminal can adjust the playback mode of the non-target music file based on a pre-set music adjustment strategy to match the playback mode of the target music file. After the playback mode is matched, the terminal can stop playing the non-target music file and play the target music file.

[0123] The music adjustment strategy can include at least one of the following: adjusting the rhythm parameters of the non-target music file, adjusting the volume of the non-target music file, and adjusting the pitch of the non-target music file. Specifically, the terminal can adjust the bpm of the non-target music file to be close to or consistent with the bpm of the target music file using a time-stretching algorithm based on a phase vocoder; alternatively, the server can dynamically adjust the volume of different frequency bands using a multi-band equalizer; alternatively, the terminal can extract the harmonic features of the music using Constant-Q transform and determine the segments in the target music file that have similar harmonic features; alternatively, the terminal can detect the beat position of the non-target music file based on an autocorrelation function and determine the cut-in point of the target music file based on the beat position; alternatively, the terminal can use sidechain compression technology to ensure that important elements such as drum beats remain clear during the adjustment process.

[0124] In step S106, in response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, a warning is issued to the target user.

[0125] In one optional implementation, if the target user's cycling speed meets a preset speed condition, or their heart rate meets a preset heart rate condition, it indicates that the target user's cycling state is relatively dangerous. Therefore, in order to reduce the possibility of the target user suffering accidental injury, in this step, the terminal can issue a warning to the target user to prompt them to reduce their cycling speed. Optionally, the terminal can issue the warning to the user in various existing ways, such as broadcasting a warning message or displaying warning text, etc., and this embodiment is not limited to these methods.

[0126] Meanwhile, in step S102, the terminal can adjust the target rhythm parameter determined in the previous polling cycle, i.e., the first rhythm parameter, to a second rhythm parameter that is smaller than the first rhythm parameter, in order to help the target user reduce the riding speed.

[0127] For example, if the target user's target rhythm parameter in the previous polling cycle was 120-140, and the target user's cycling speed in the current polling cycle meets the preset speed condition, and / or heart rate meets the preset heart rate condition, the terminal can adjust the target rhythm parameter for the current cycle to 100-120 and issue a warning to the target user.

[0128] Preset speed and heart rate conditions can be set according to actual needs. For example, the preset speed condition can be that the target user's cycling speed is greater than 40 km / h, and the preset heart rate condition can be that the target user's heart rate is greater than 170 beats / minute.

[0129] In this embodiment of the invention, after acquiring the user's walking speed and heart rate, the rhythm parameters of the music to be played are determined based on these parameters. Candidate music files are then selected from a preset music file library based on these rhythm parameters, and finally, the target music file to be played is chosen from the candidate files. In this embodiment, each music file in the preset music file library is generated based on a pre-trained music generation model. This music generation model is trained using sample audio files, instrument digital interface files of the sample audio files, attribute information, and note sequences. Therefore, it increases the richness of the music and improves the flexibility of music playback, thereby enhancing the user experience.

[0130] Figure 6 This is a schematic diagram of a music playback device according to an embodiment of the present invention. Figure 6 As shown, the music playback device in this embodiment includes a data acquisition unit 601, a parameter acquisition unit 602, a matching unit 603, a selection unit 604, and a playback unit 605.

[0131] The data acquisition unit 601 is used to acquire the walking speed and heart rate of the target user; the parameter acquisition unit 602 is used to determine the target rhythm parameter based on the walking speed and heart rate; the matching unit 603 is used to determine candidate music files from a preset music file library based on the target rhythm parameter, wherein each preset music file in the preset music file library is obtained based on a pre-trained music generation model, and the pre-trained music generation model is trained based on a training sample set, wherein the training sample set includes multiple sample audio files, instrument digital interface files of each sample audio file, attribute information, and note sequences, wherein the attribute information includes the rhythm parameter of the sample audio file; the selection unit 604 is used to select the target music file from the candidate music files; and the playback unit 605 is used to play the target music file.

[0132] Furthermore, the parameter acquisition unit 602 includes a first parameter acquisition subunit. The first parameter acquisition subunit is used to determine the target rhythm parameter based on a pre-set mapping relationship between travel speed, heart rate, and rhythm parameters, and the travel speed and heart rate.

[0133] Furthermore, the playback unit 605 includes an adjustment subunit and a playback subunit.

[0134] The adjustment subunit is configured to, in response to the presence of a non-target music file being played, adjust the playback mode of the non-target music file based on a pre-set music adjustment strategy to match the target music file. The music adjustment strategy includes at least one of adjusting the rhythm parameter of the non-target music file, adjusting the volume of the non-target music file, and adjusting the pitch of the non-target music file. The playback subunit is configured to, in response to the playback mode matching the target music file, stop playing the non-target music file and play the target music file.

[0135] Furthermore, the music generation model includes a feature embedding module, an encoding module, a feature fusion module, a conditional layer, and a conditional decoding module. The training sample set also includes the timbre tags of the sample audio files. The music generation model is trained through a file determination unit, a first feature acquisition unit, a sequence acquisition unit, a second feature acquisition unit, a third feature acquisition unit, and a training unit.

[0136] The system comprises the following components: a file determination unit for identifying each sample audio file as a specified audio file; a first feature acquisition unit for determining the input of the feature embedding module based on the specified audio file and acquiring the timbre embedding features of the specified audio file; a sequence acquisition unit for determining the input of the encoding module based on the instrument digital interface file and acquiring the hidden state sequence of the specified audio file; a second feature acquisition unit for determining the input of the conditional layer based on the attribute information of the specified audio file and acquiring the sample conditional embedding features of the specified audio file; a third feature acquisition unit for determining the input of the feature fusion module based on the timbre embedding features and the hidden state sequence and acquiring the fusion features of the specified audio file; and a training unit for training the music generation model using the fusion features and the sample conditional embedding features as input to the conditional decoding module and the note sequence and the timbre labels as training targets, until the loss function of the music generation model converges.

[0137] Furthermore, the first feature acquisition unit includes a first acquisition subunit, a first conversion subunit, and a second acquisition subunit.

[0138] The first acquisition subunit is used to acquire the spectrogram of the specified audio file; the first conversion subunit is used to convert the spectrogram into the frequency cepstral coefficient features of the specified audio file; and the second acquisition subunit is used to acquire the timbre embedding features by using the frequency cepstral coefficient features as input to the feature embedding module.

[0139] Furthermore, the music generation model includes a conditional layer and a conditional decoding module;

[0140] The preset music file is generated by a condition acquisition unit, a fourth feature acquisition unit, a sequence acquisition unit, and a conversion unit.

[0141] The system includes a condition acquisition unit for acquiring music generation conditions, which are determined based on the target user's preference information; a fourth feature acquisition unit for determining the input of the condition layer based on the music generation conditions and acquiring the condition embedding features of the preset music file; a sequence acquisition unit for using the condition embedding features as the input of the condition decoding module to acquire the target note sequence of the preset music file; and a conversion unit for converting the target note sequence into the preset music file.

[0142] Furthermore, the conversion unit includes an optimization subunit and a second conversion subunit.

[0143] The optimization subunit is used to optimize the target note sequence based on a pre-set optimization algorithm to obtain the optimized target note sequence; the second conversion subunit is used to convert the optimized target note sequence into the preset music file.

[0144] Furthermore, the parameter acquisition unit 602 includes a parameter adjustment subunit.

[0145] The parameter adjustment subunit is used to adjust the target rhythm parameter from a first rhythm parameter to a second rhythm parameter in response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, so as to update the target music file, wherein the first rhythm parameter is greater than the second rhythm parameter.

[0146] Furthermore, the device also includes a warning unit.

[0147] The warning unit is used to issue a warning to the target user in response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, so as to prompt the target user to reduce the travel speed.

[0148] In this embodiment of the invention, after acquiring the user's walking speed and heart rate, the rhythm parameters of the music to be played are determined based on these parameters. Candidate music files are then selected from a preset music file library based on these rhythm parameters, and finally, the target music file to be played is chosen from the candidate files. In this embodiment, each music file in the preset music file library is generated based on a pre-trained music generation model. This music generation model is trained using sample audio files, instrument digital interface files of the sample audio files, attribute information, and note sequences. Therefore, it increases the richness of the music and improves the flexibility of music playback, thereby enhancing the user experience.

[0149] Figure 7 This is a schematic diagram of an electronic device according to an embodiment of the present invention. (For example...) Figure 7 As shown, electronic device 7 is a general-purpose data processing device, which includes a general-purpose computer hardware structure, including at least a processor 701 and a memory 702. The processor 701 and memory 702 are connected via a bus 703. The memory 702 is adapted to store instructions or programs executable by the processor 701. The processor 701 can be a standalone microprocessor or a collection of one or more microprocessors. Thus, the processor 701 executes the instructions stored in the memory 702 to perform the method flow of the embodiments of the present invention as described above, thereby realizing data processing and control of other devices. The bus 703 connects the aforementioned components together, and also connects the aforementioned components to a display controller 704, a display device, and an input / output (I / O) device 7070. The input / output (I / O) device 7070 can be a mouse, keyboard, modem, network interface, touch input device, motion-sensing input device, printer, and other devices known in the art. Typically, the input / output device 7070 is connected to the system via an input / output (I / O) controller 706.

[0150] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus (devices), or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0151] This application is described with reference to flowchart illustrations of methods, apparatus (devices), and computer program products according to embodiments of this application. It should be understood that each step in the flowchart can be implemented by computer program instructions.

[0152] These computer program instructions may be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including an instruction means, the implementation process of which is described in the instruction means. Figure 1 The function specified in one or more processes.

[0153] These computer program instructions may also be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing device, produce instructions for implementing processes. Figure 1 A device for a function specified in one or more processes.

[0154] Another embodiment of the present invention relates to a non-volatile storage medium for storing a computer-readable program for use by a computer to execute some or all of the above-described method embodiments.

[0155] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program specifying the relevant hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0156] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. For those skilled in the art, the present invention can be modified and varied in various ways. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principle of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for playing music, characterized in that, The method includes: Obtain the target user's walking speed and heart rate; The target rhythm parameters are determined based on the travel speed and heart rate. Candidate music files are determined from a preset music file library based on the target rhythm parameters. Each preset music file in the preset music file library is obtained based on a pre-trained music generation model. The pre-trained music generation model is trained based on a training sample set. The training sample set includes multiple sample audio files, instrument digital interface files of each sample audio file, attribute information, and note sequences. The attribute information includes the rhythm parameters of the sample audio files. Select the target music file from the candidate music files; Play the target music file.

2. The method according to claim 1, characterized in that, The determination of the target rhythm parameters based on the travel speed and heart rate includes: The target rhythm parameter is determined based on the pre-set mapping relationship between walking speed, heart rate, and rhythm parameters, and the walking speed and heart rate.

3. The method according to claim 1, characterized in that, Playing the target music file includes: In response to the presence of a non-target music file in playback, the playback mode of the non-target music file is adjusted based on a pre-set music adjustment strategy to match the target music file. The music adjustment strategy includes at least one of adjusting the rhythm parameter of the non-target music file, adjusting the volume of the non-target music file, and adjusting the pitch of the non-target music file. In response to the playback method matching the target music file, playback of the non-target music file is stopped, and the target music file is played.

4. The method according to claim 1, characterized in that, The music generation model includes a feature embedding module, an encoding module, a feature fusion module, a conditional layer, and a conditional decoding module. The training sample set also includes the timbre tags of the sample audio files. The music generation model is trained in the following manner: Each of the sample audio files is designated as a specific audio file; The input to the feature embedding module is determined based on the specified audio file, and the timbre embedding features of the specified audio file are obtained. The input to the encoding module is determined based on the instrument digital interface file, and the hidden state sequence of the specified audio file is obtained; The input to the conditional layer is determined based on the attribute information of the specified audio file, and the sample conditional embedding features of the specified audio file are obtained. The input to the feature fusion module is determined based on the timbre embedding features and the hidden state sequence, and the fusion features of the specified audio file are obtained. The fusion features and the sample conditional embedding features are used as inputs to the conditional decoding module, and the music generation model is trained using the note sequence and the timbre label as training targets until the loss function of the music generation model converges.

5. The method according to claim 4, characterized in that, The step of determining the input of the feature embedding module based on the specified audio file and obtaining the timbre embedding features of the specified audio file includes: Obtain the spectrogram of the specified audio file; Convert the spectrogram into the frequency cepstral coefficient features of the specified audio file; The timbre embedding feature is obtained by using the frequency cepstral coefficient feature as the input of the feature embedding module.

6. The method according to claim 1, characterized in that, The music generation model includes a conditional layer and a conditional decoding module; The preset music file is generated in the following way: Obtain music generation conditions, which are determined based on the target user's preference information; The input to the conditional layer is determined based on the music generation conditions, and the conditional embedding features of the preset music file are obtained. Using the conditional embedding features as input to the conditional decoding module, the target note sequence of the preset music file is obtained; The target note sequence is converted into the preset music file.

7. The method according to claim 6, characterized in that, The step of converting the target note sequence into the preset music file includes: The target note sequence is optimized based on a pre-set optimization algorithm to obtain the optimized target note sequence; The optimized target note sequence is converted into the preset music file.

8. The method according to claim 1, characterized in that, The determination of the target rhythm parameters based on the travel speed and heart rate includes: In response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, the target rhythm parameter is adjusted from a first rhythm parameter to a second rhythm parameter to update the target music file, wherein the first rhythm parameter is greater than the second rhythm parameter.

9. The method according to claim 1, characterized in that, The method further includes: In response to the travel speed meeting a preset speed condition and / or the heart rate meeting a preset heart rate condition, a warning is issued to the target user to prompt the target user to reduce the travel speed.

10. A music playback device, characterized in that, The device includes: The data acquisition unit is used to acquire the target user's walking speed and heart rate; The parameter acquisition unit is used to determine the target rhythm parameters based on the travel speed and heart rate. A matching unit is used to determine candidate music files from a preset music file library according to the target rhythm parameters. Each preset music file in the preset music file library is obtained based on a pre-trained music generation model. The pre-trained music generation model is trained based on a training sample set. The training sample set includes multiple sample audio files, instrument digital interface files of each sample audio file, attribute information, and note sequences. The attribute information includes the rhythm parameters of the sample audio files. The selection unit is used to select a target music file from the candidate music files; The playback unit is used to play the target music file.

11. A computer program product, characterized in that, The computer program product includes a computer program / instruction that, when executed by a processor, implements the method of any one of claims 1-9.

12. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method of any one of claims 1-9.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1-9.

Images