Method, device, equipment, storage medium and program product for generating music accompaniment

Abstract

The present application discloses a method, device, equipment, storage medium and program product for generating music accompaniment, and relates to the field of computer technology. The method comprises: obtaining music data; extracting music information from the music data to obtain beat data information, chord data information and melody data information; using the melody data information as a generation restriction condition for the music accompaniment, and generating music accompaniment data through the beat data information and chord data information; rendering audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data. In the above manner, it is possible to quantize the music beat and music notes, incorporate the beat and chord into the accompaniment melody generation process within the limitation of the music melody, so that the music accompaniment data can be presented more meticulously through music parameters, thereby improving the generation stability and generation effect of the music accompaniment. The present application can be applied to various scenarios such as music accompaniment generation.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a method, apparatus, device, storage medium, and program product for generating a music accompaniment. Background Art

[0002] Artistic creation has always been considered the exclusive domain of artists. However, with the development of computer technology in recent years, artistic creation has gradually broken through traditional barriers. With the help of computer technology, musical works can be artificially produced, giving music types and content greater room for improvement.

[0003] In related technologies, if one wishes to generate corresponding musical accompaniment based on music, a deep learning-based method is usually used to generate the accompaniment. The model is trained with the help of sample music and the corresponding sample accompaniment so that the model learns the method of generating the accompaniment. The music for which the accompaniment needs to be generated is then input into the trained model to obtain the musical accompaniment.

[0004] The above process relies too much on the accurate correspondence between the sample music and the sample accompaniment. Although this method can obtain good music accompaniment to a certain extent, it ignores the music theory information of the music itself, which easily leads to a large gap between the style of the music accompaniment and the music itself, affecting the accuracy of the music accompaniment in the music field. Summary of the Invention

[0005] The present application provides a method, apparatus, device, storage medium, and program product for generating a musical accompaniment. These methods can quantize the beats and notes of a musical piece, incorporating beats and chords into the accompaniment melody generation process within the constraints of the musical melody. This allows the musical accompaniment data to be presented more meticulously through musical parameters, thereby improving the stability and quality of musical accompaniment generation. The technical solution is as follows.

[0006] In one aspect, a method for generating a music accompaniment is provided, the method comprising:

[0007] Acquire music data, where the music data is divided into a plurality of music beats, and the music beats include music notes;

[0008] Extracting music information from the music data to obtain beat data information, chord data information, and melody data information, wherein the beat data information is used to characterize the changing speed of the multiple music beats, the chord data information is used to describe the chord units extracted based on the music beats, and the melody data information is used to describe the note changes between the multiple music notes;

[0009] The melody data information is used as a generation restriction condition for the music accompaniment, and music accompaniment data is generated by using the beat data information and the chord data information, wherein the melody data information is used to limit the accompaniment melody of the music accompaniment by the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data by using music parameters;

[0010] Audio data rendering is performed based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

[0011] In another aspect, a device for generating a music accompaniment is provided, the device comprising:

[0012] A data acquisition module is used to acquire music data, where the music data is divided into a plurality of music beats, and the music beats include music notes;

[0013] an information extraction module, configured to extract music information from the music data to obtain beat data information, chord data information, and melody data information, wherein the beat data information is used to characterize the changing speed of the multiple music beats, the chord data information is used to describe the chord units extracted based on the music beat, and the melody data information is used to describe the note changes between the multiple music notes;

[0014] a data generation module, configured to use the melody data information as a generation restriction condition for the music accompaniment, and to generate music accompaniment data using the beat data information and the chord data information, wherein the melody data information is used to limit the accompaniment melody of the music accompaniment using the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data using music parameters;

[0015] The accompaniment generation module is used to render audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

[0016] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by the processor to implement the method for generating a musical accompaniment as described in any of the above-mentioned embodiments of the present application.

[0017] On the other hand, a computer-readable storage medium is provided, wherein the storage medium stores at least one instruction, at least one program, a code set, or an instruction set, and the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the method for generating a musical accompaniment as described in any of the above-mentioned embodiments of the present application.

[0018] In another aspect, a computer program product or computer program is provided. The computer program product or computer program includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for generating a musical accompaniment described in any of the above embodiments.

[0019] The beneficial effects of the technical solutions provided in the embodiments of the present application include at least:

[0020] The obtained music data is subjected to music information extraction to obtain beat data information, chord data information, and melody data information; music accompaniment data is generated through the beat data information and chord data information with the melody data information generating restriction condition, and then the music accompaniment is obtained based on the music accompaniment data rendering. In the process of generating music accompaniment data from the music data, the music beat and music notes are quantized through the music information extraction process to improve the accuracy of the analysis of the music data, making the accompaniment data generation process more targeted; then, the melody data information, beat data information, and chord data information are integrated, and the beat and chord are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more meticulously through music parameters, and then, through the music accompaniment data rendering, a music accompaniment that can more accurately present the music data is obtained, thereby improving the generation stability and generation effect of the music accompaniment. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 This is a schematic diagram of an implementation environment provided by an exemplary embodiment of the present application;

[0023] Figure 2 is a flowchart of a method for generating a music accompaniment provided by an exemplary embodiment of the present application;

[0024] Figure 3 is a flowchart of a method for generating a music accompaniment provided by another exemplary embodiment of the present application;

[0025] Figure 4 is a flowchart of a method for generating a music accompaniment provided by yet another exemplary embodiment of the present application;

[0026] Figure 5is a flowchart of a method for generating a music accompaniment provided by another exemplary embodiment of the present application;

[0027] Figure 6 1 is a schematic diagram of the overall framework of a method for generating a music accompaniment provided by an exemplary embodiment of the present application;

[0028] Figure 7 is an input schematic diagram of an accompaniment generation method provided by an exemplary embodiment of the present application;

[0029] Figure 8 is a processing flow chart of a method for generating a music accompaniment provided by an exemplary embodiment of the present application;

[0030] Figure 9 is a schematic diagram of piano notation provided by an exemplary embodiment of the present application;

[0031] Figure 10 This is a structural block diagram of a device for generating a music accompaniment provided by an exemplary embodiment of the present application;

[0032] Figure 11 This is a structural block diagram of a server provided by an exemplary embodiment of the present application. DETAILED DESCRIPTION

[0033] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0034] First, a brief introduction is given to the terms involved in the embodiments of this application.

[0035] Symbolic domain: In music theory, the symbolic domain refers to the range of symbols or values ​​used by computers when processing music. Specifically, the symbolic domain refers to the digital music format that computers read and process during the generation process. In digital music formats, musical elements such as notes, pitch, and volume are typically represented digitally. Examples of digital music formats include the Musical Instrument Digital Interface (MIDI) format and the Music Extensible Markup Language (MusicXML) format. For example, MIDI uses numbers to represent notes, pitch, volume, and other musical parameters. In MIDI files, different numerical values ​​represent different notes, and the velocity of a note can be represented numerically. The symbolic domain may include note value (a number representing the duration of a note, such as 4 for a quarter note and 8 for an eighth note), pitch (a number or code representing the pitch of a note; in MIDI, each note has a corresponding pitch value), and volume (a number representing the volume level of a note or instrument, with larger values ​​typically indicating a higher volume). These numbers or symbols constitute the symbolic domain used by computers in the music generation process. Through these symbols, computers can read, process and generate music. In this context, the definition of symbolic domain refers to the range and way in which musical elements can be represented digitally.

[0036] Piano cover: refers to the adaptation and performance of the original song in audio format with vocals and complete accompaniment in the form of piano music.

[0037] In related technologies, if one wishes to generate a corresponding musical accompaniment based on music, a deep learning-based approach is typically used. A model is trained using sample music and the corresponding sample accompaniment, allowing the model to learn how to generate the accompaniment. The music for which the accompaniment is to be generated is then input into the trained model to generate the musical accompaniment. This process relies heavily on the accurate correspondence between the sample music and the sample accompaniment. While this approach can produce good musical accompaniment to a certain extent, it ignores the musical theory inherent in the music itself, which can easily lead to a significant discrepancy between the musical style of the musical accompaniment and the music itself, affecting the accuracy of the musical accompaniment in the music field.

[0038] In an embodiment of the present application, a method for generating a music accompaniment is introduced, which can quantize the music beat and music notes through the music information extraction process, thereby improving the accuracy of the analysis of the music data; and then integrate multiple quantitative information, and incorporate the beat and chords into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, thereby improving the generation stability and generation effect of the music accompaniment. The method for generating a music accompaniment can be applied to the fields of music production, music creation, game production, advertising production, virtual reality (VR) and augmented reality (AR), etc., and the embodiments of the present application are not limited to this.

[0039] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, storage, display, etc.), and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data must comply with the relevant laws, regulations, and standards of the relevant region. For example, the music data and music information extraction involved in this application were obtained with full authorization.

[0040] Next, the implementation environment involved in the embodiments of the present application is described. The method for generating a musical accompaniment provided in the embodiments of the present application can be implemented by a terminal alone, by a server, or by a terminal and a server through data interaction, and the embodiments of the present application are not limited thereto. Optionally, the method for generating a musical accompaniment executed interactively by a terminal and a server is used as an example for description.

[0041] For illustration, please refer to Figure 1 The implementation environment involves a terminal 110 and a server 120 , and the terminal 110 and the server 120 are connected via a communication network 130 .

[0042] In some embodiments, the terminal 110 has a music data acquisition function, and can obtain music data by recording, manual production, etc. For example, a song is used as music data.

[0043] Optionally, the terminal 110 sends the music data to the server 120 via the communication network 130, and the server 120 obtains the music data. The music data is divided into a plurality of music beats, and the music beats include music notes.

[0044] In some embodiments, the server 120 extracts music information from the music data to obtain beat data information, chord data information, and melody data information.

[0045] Among them, the beat data information is used to represent the changing speed of multiple music beats, the chord data information is used to describe the chord units extracted based on the music beat, and the melody data information is used to describe the note changes between multiple music notes.

[0046] In some embodiments, the server 120 uses the melody data information as a constraint condition for generating the music accompaniment, and generates the music accompaniment data through the beat data information and the chord data information.

[0047] The melody data information is used to define the accompaniment melody of the music accompaniment based on the music melody of the music data. The music accompaniment data describes the accompaniment of the music data through musical parameters. As the melody data represents the entire music data, it helps to play an overall limiting role in the music accompaniment generation process. Based on the accompaniment melody defining the accompaniment melody, the beat data information represents the tempo and the chord data information represents the local notes of the music, thereby generating more accurate music accompaniment data.

[0048] In some embodiments, the server 120 renders the audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

[0049] The music accompaniment data, as the accompaniment described by music parameters, is music data content presented in the form of a symbolic domain. Therefore, to obtain the music accompaniment corresponding to the music data, it is necessary to render the music accompaniment data into audio data to obtain the played music accompaniment. For example, the music accompaniment data can be analyzed using specific audio data decoding software to obtain the music accompaniment represented by the audio data of the music accompaniment data.

[0050] Optionally, the server 120 sends the music accompaniment data to the terminal 110 via the communication network, so that the terminal 110 renders the music accompaniment data into audio data to obtain the music accompaniment; or, the server renders the music accompaniment data into audio data to obtain an accompaniment data file that is convenient for the terminal to directly play, and sends the accompaniment data file to the terminal 110 via the communication network, and the terminal 110 downloads and plays the music accompaniment based on the accompaniment data file.

[0051] It is worth noting that the above-mentioned terminals include but are not limited to mobile terminals such as mobile phones, tablet computers, portable laptops, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, etc., and can also be implemented as desktop computers, etc.; the above-mentioned servers can be independent physical servers, or server clusters or distributed systems composed of multiple physical servers, or cloud servers.

[0052] In some embodiments, the above-mentioned server can also be implemented as a node in a blockchain system.

[0053] Combined with the above-mentioned noun introduction and application scenarios, the method for generating music accompaniment provided by this application is described, and the method is applied to the server as an example. Figure 2 As shown, the method includes the following steps 210 to 240.

[0054] Step 210: Acquire music data.

[0055] Schematically, music data refers to data expressed in audio form, and is used to contain the actual audio content of music.

[0056] Optionally, the music data can be stored in formats such as Waveform Audio File Format (WAV), Moving Picture Experts Group Audio Layer III (MP3), and Free Lossless Audio Codec (FLAC). Since the music data contains sound wave signals, the music data can be used to play music content.

[0057] For example, music data may be implemented as a vocal song, such as an album of music sung by a singer, or a song sung by an individual, or a song hummed by an individual, etc.; music data may also be implemented as a radio station program, a television program, a news broadcast, etc.; music data may also be implemented as a concert recording or a concert recording; music data may also be implemented as a telephone recording, etc.

[0058] The music data is divided into a plurality of music beats, and the music beats include music notes.

[0059] Meter is a fundamental organizing element in music, determining the temporal arrangement and emphasis of musical notes. As a means of expressing a sense of musical time, a piece of music typically consists of alternating strong and weak beats. The strong and weak relationships between beats create a rhythmic sense of musical data. The use and emphasis of beats vary across musical styles and cultures. Meter is a crucial means for composers to express emotion and convey musical information.

[0060] Schematically, a musical beat is a unit of time within a time signature. Each beat has a strong beat (usually the first beat), with the remaining beats being weak beats. A beat, a symbol used on musical notation to indicate rhythm, consists of two numbers: the upper number represents the number of beats per measure, and the lower number represents the duration of each beat. The duration represents the duration of a musical note within the beat. For example, a 4 / 4 time signature indicates four beats per measure, with each beat being the duration of a quarter note.

[0061] In other words, a musical beat is a basic unit of time in music, and the notes contained within it determine the musical information within this time unit. Within a beat, notes of different durations can be accommodated, and the combination and arrangement of these notes form the rhythm of the music.

[0062] Step 220: extract music information from the music data to obtain beat data information, chord data information, and melody data information.

[0063] Illustratively, music information extraction is used to quantize music data so as to present the music data presented in audio form in data form.

[0064] Optionally, music information extraction is performed on the music data to obtain beat data information representing rhythm changes of the music data, chord data information representing different chords in the music data, and melody data information representing overall melody changes of the music data.

[0065] That is, the beat data information is used to characterize the changing speed of multiple music beats, the chord data information is used to describe the chord units extracted based on the music beat, and the melody data information is used to describe the note changes between multiple music notes.

[0066] Schematically, a musical beat, as the basic time unit in musical data, creates the basic rhythmic structure of the musical data and defines the ordering of musical notes on the timeline. When represented by a beat (Time Signature), a musical beat specifies the number of beats in each measure and the duration of each beat. When acquiring beat data information, one can first determine the multiple beats corresponding to the musical data, and then determine the beat data information based on the distribution changes of the multiple beats.

[0067] Schematically, a chord is a collection of at least two notes played simultaneously, arranged in a fixed interval relationship. Chords are the basis of harmony in music and are very important for creating and emphasizing a sense of harmony. A chord can contain musical notes such as a fundamental tone, a third tone, and a fifth tone, thereby forming different harmonic textures. In other words, chords focus on the combination of sounds that occur simultaneously, and they focus on expressing pitch relationships in the vertical direction. When acquiring chord data information, the musical notes in each musical beat can be determined in units of musical beats, and the musical audio can be grouped into chords (or chord units).

[0068] Illustratively, a melody is a series of organically connected notes, played in chronological order, forming a musical expression with melodic clues. Melody is a combination of pitch and duration, and is the most recognizable and memorable part of music. It typically consists of a primary melody and a secondary melody. In other words, melody focuses on the horizontal combination of pitch and duration. When acquiring melody data information, attention is paid to the pitch changes corresponding to multiple musical notes and the durations corresponding to multiple musical notes, thereby comprehensively determining the melody data information corresponding to the musical data.

[0069] Within a piece of music data, the three elements of beat, chords, and melody often intertwine to create a complex and engaging musical experience. Meter provides a temporal framework, chords provide a harmonic foundation, and melody provides guiding clues, allowing for a more nuanced presentation of the music data.

[0070] With the help of the music information extraction process, the music beat is quantized into beat data information, the chord is quantized into chord data information, and the melody is quantized into melody data information, so that the music data can be analyzed based on the multiple quantized information after the information extraction.

[0071] Step 230: Using the melody data information as a generation constraint for the music accompaniment, the music accompaniment data is generated through the beat data information and the chord data information.

[0072] Illustratively, after obtaining the melody data information, the melody data information is used to characterize the overall melody of the music data, and thus the melody data information is used as a generation restriction condition to generate a music accompaniment corresponding to the music data within the music melody represented by the melody data information.

[0073] That is, the melody data information is used to define the accompaniment melody of the music accompaniment using the music melody of the music data.

[0074] Schematically, a musical melody represents a series of organically connected musical notes, played in chronological order to form a musical expression with melodic clues. It is the most recognizable and memorable part of musical data. Melody data information refers to the digital, symbolic, or computer-readable representation of the melody. It can include pitch sequences, time value information, note duration, etc., and is usually encoded in digital or symbolic form. It is information used for computer analysis, music generation, or other digital music processing tasks. In other words, there is a corresponding relationship between melody data information and the musical melody, and melody data information is the digital form of the musical melody.

[0075] Similarly, an accompaniment melody is used to represent a series of organically connected musical notes, played in chronological order to form a musical accompaniment with a melodic clue. Therefore, when melody data information is used as a constraint condition for generating a musical accompaniment, the musical melody corresponding to the melody data information is used to limit the accompaniment melody corresponding to the musical accompaniment.

[0076] The music accompaniment data describes the accompaniment of the music data through music parameters.

[0077] Schematically, there is a correspondence between the music accompaniment data and the accompaniment melody, and the music accompaniment data is the digital form of the accompaniment melody; when generated through beat data information and chord data information, the music accompaniment data corresponding to the accompaniment melody is generated. The music accompaniment data, as a digital form corresponding to the accompaniment melody, is the content generated after describing the accompaniment of the music data through numerous music parameters.

[0078] Because music accompaniment is not only related to the melody of the music data, but also to the chords and beats, when generating music accompaniment data, it is expressed not only through musical parameters related to the melody, such as pitch, duration, and timbre, but also through musical parameters related to the chords, such as pitch and harmony, and also through musical parameters related to the beat, such as volume and rhythmic pattern. In other words, the musical parameters used to describe the accompaniment of the music data include various parameters related to the music accompaniment, such as pitch, timbre, volume, rhythmic pattern, and harmony.

[0079] Step 240 , performing audio data rendering based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

[0080] Optionally, after obtaining the music accompaniment data, since the music accompaniment data is in digital form corresponding to the music accompaniment, the music accompaniment data can be decoded to present the music accompaniment data in audio form. In other words, the music accompaniment data is rendered as audio data to obtain the music accompaniment corresponding to the music data.

[0081] Illustratively, music accompaniment data is implemented as a digital representation of the music data—a MIDI file. To obtain the music accompaniment corresponding to the music accompaniment data, MIDI files contain musical information such as notes, pitches, and durations, but do not inherently contain sound waveforms. Therefore, the MIDI files must be converted into accompaniment data for rendering the music accompaniment. Specifically, audio conversion is first performed based on the music accompaniment data to obtain the accompaniment audio; then, audio rendering is performed on the accompaniment audio to obtain the music accompaniment.

[0082] Optionally, the method of performing audio conversion on music accompaniment data and obtaining accompaniment data includes the following steps.

[0083] (1) Software tools: Use specialized music production software or MIDI editors to convert MIDI files into accompaniment data. MIDI editors include professional sequencers (Ableton Live), Fruity Loops Studio (FL Studio), Logic Pro, Fluid Synth, Timidity++, etc.

[0084] (2) Virtual instruments or sound sources: Select or load virtual instrument or sound source plug-ins in the above software. These plug-ins act as virtual instruments and can generate corresponding audio waveforms based on the information in the MIDI file.

[0085] (3) Connect MIDI files: Import MIDI files into the software, usually by dragging and dropping files or using the import function. MIDI files contain information such as musical notes, pitches, and durations.

[0086] (4) Assigning a timbre: Assigning a suitable timbre (instrument) to a MIDI file or note determines which instrument is used to simulate the note in the MIDI file. This process is usually completed in the interface of a virtual instrument or sound source plug-in.

[0087] (5) Render to audio: Use the "Export" or "Render" function of the preset software to render the MIDI file into an audio file, usually WAV, MP3, Audio Interchange File Format (AIFF), etc. The preset software will generate the corresponding audio waveform based on the MIDI file.

[0088] (6) Adjust the effect: You can add audio effects, reverb, equalizer, etc. in the software as needed to adjust the final effect of the audio.

[0089] It's worth noting that the sound quality and expressiveness of the MIDI file-to-accompaniment audio conversion process described above depend on the quality of the virtual instrument or sound source used. Some professional virtual instrument libraries offer high-quality sound and realistic audio generation. This process converts the musical information in the MIDI file into a computer-readable audio format (accompaniment audio), allowing you to play the accompaniment audio on a computer or other audio device.

[0090] Optionally, the accompaniment audio after the audio conversion is rendered to obtain the music accompaniment, for example, the accompaniment audio is opened through a terminal to play the music accompaniment.

[0091] It should be noted that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0092] In summary, music information is extracted from the acquired music data to obtain beat data information, chord data information, and melody data information; music accompaniment data is generated through beat data information and chord data information with the melody data information generating restriction condition, and then music accompaniment is obtained based on the music accompaniment data rendering. In the process of generating music accompaniment data from music data, the music beat and music notes are quantized through the music information extraction process to improve the accuracy of the analysis of the music data, making the accompaniment data generation process more targeted; then, the melody data information, beat data information, and chord data information are integrated, and the beat and chord are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more meticulously through music parameters, and then, through the music accompaniment data rendering, a music accompaniment that can more accurately present the music data is obtained, thereby improving the generation stability and generation effect of the music accompaniment.

[0093] In an optional embodiment, the beat data information is determined by the number of multiple music beats in a unit time, and the melody data information and chord data information corresponding to the music data are obtained through the melody track and the chord track. Figure 3 As shown above Figure 2 The illustrated step 220 may also be implemented as the following steps 310 to 330 .

[0094] Step 310 : Determine beat data information corresponding to the music data based on the number of multiple music beats within a unit time.

[0095] Indicatively, the unit time is a pre-set unit time length, for example, the unit time is 1 minute, or the unit time is half a minute, or the unit time is 5 seconds, etc.

[0096] Optionally, beats per minute (BPM) is used as the data tempo information corresponding to the music data; BPM is a unit used to indicate tempo in music, representing the number of beats per minute. In determining the BPM, manual calculation can be used, or tools such as music production software can be used. For example, when manually calculating, if you can feel the rhythm of the music, you can use a watch or timer to determine the number of beats within a minute. For example, if you can count 120 beats within a minute, the BPM is 120. Alternatively, a music beat detector can be used to determine the BPM, which can display the song's BPM.

[0097] Step 320: extract music information from the music data to determine the chord track and melody track corresponding to the music data.

[0098] The chord track is used to describe a sound group composed of multiple music notes with a time sequence relationship in the form of chords.

[0099] Schematically, a chord track is a feature in music production software that is used to represent the chord progression in music.

[0100] Optionally, the Chord track is often divided into measures to better organize and understand the structure of the music. The number of measures on the Chord track depends on the structure and arrangement of the music; in music production software, chords can be created and edited on the Chord track, with each measure typically corresponding to a measure in the music. This helps maintain a consistent chord progression throughout the track, making it easier to adjust and organize the music.

[0101] The melody track is used to describe the main melody composed of multiple music notes in a time sequence.

[0102] For example, in music production software, a melody track is a concept used in music production. It refers to the audio track or track containing the main melody. The main melody is the most prominent and eye-catching melodic line in the music, usually performed by an instrument or vocal performance (such as a human voice). In music production software, producers use the melody track to record, edit, and arrange the main melody.

[0103] The melody track, which contains the main melody, can capture the most important and memorable melodic line in a piece of music. This is because the main melody is one of the key elements in music that captures the audience's attention. The melody track allows producers to precisely shape and control the core elements of the music, creating works with unique style and emotion.

[0104] Optionally, the Chord track is usually divided into bars. The number of bars in the Melody track depends on the structure of the music, the arrangement, and the creative style of the producer.

[0105] Step 330: Acquire melody data information and chord data information corresponding to the music data based on the melody track and the chord track.

[0106] In an optional embodiment, the number of bars in the melody track and the number of bars in the chord track are determined respectively to obtain a first number corresponding to the melody track and a second number corresponding to the chord track.

[0107] The first number is used to represent the number of bars in the melody track, that is, the melody track includes the first number of bars; the second number is used to represent the number of bars in the chord track, that is, the chord track includes the second number of bars.

[0108] Illustratively, the melody track and the chord track are usually each divided into bars; after the melody track is divided into bars, a first number of bars are obtained; after the chord track is divided into bars, a second number of bars are obtained.

[0109] Optionally, a music feature extraction model is pre-acquired. The music feature extraction model is a pre-trained model used to analyze features of the music data, such as the beat, chords, and melody. Bar extraction is performed on the melody track and the chord track using the music feature extraction model, thereby obtaining a first number of bars in the melody track and a second number of bars in the chord track.

[0110] Illustratively, the first quantity and the second quantity may be equal; or, the first quantity and the second quantity may be different, such as: the first quantity is smaller than the second quantity, or the second quantity is smaller than the first quantity, etc.

[0111] In an optional embodiment, the second number of chord tracks is adjusted based on the first number, and the adjusted chord tracks are obtained as the chord data information.

[0112] Optionally, in response to the second number being less than the first number, the chord track is padded based on the first number to obtain a padded chord track as the chord data information. The padded chord track is the adjusted chord track.

[0113] Schematically, if the second number is smaller than the first number, it means that the number of bars obtained after dividing the chord track into bars is smaller, and the number of bars obtained after dividing the melody track into bars is larger; based on the fact that the melody track can better represent the overall effect of the music data, the second number is padded based on the first number corresponding to the melody track, thereby obtaining the padded chord track as the chord data information.

[0114] Illustratively, during the filling operation, the chord of the measure preceding the measure to be filled in the chord track is copied to obtain the filled chord track as the chord data information.

[0115] In some embodiments, each measure in the completed chord track is checked. To avoid long pauses in the accompaniment, when there is a measure without any chords, the chords of the previous measure are copied; if the measure is the first measure, the chords of the first subsequent measure that is not a full rest are copied.

[0116] Optionally, in response to the second number being equal to the first number, the chord track is used as the chord data information; in response to the second number being greater than the first number, the melody track and the chord track are aligned to obtain an aligned chord track corresponding to the melody track as the chord data information, etc.

[0117] In an optional embodiment, based on the pitches of the notes corresponding to the multiple music notes in the melody track, a melody transformation process is performed on the melody track to obtain an updated melody track as the melody data information.

[0118] Schematically, the melody track describes the note characteristics corresponding to multiple music notes. Pitch is a basic and important concept in music, which determines the relative height of the music notes.

[0119] Optionally, for some songs whose melodies generally have a lower pitch, in order to improve the listening experience and prevent conflicts with the accompaniment, the melody can be selectively processed to a higher octave.

[0120] In some embodiments, a preset low pitch and a preset high pitch are obtained.

[0121] The preset low pitch and the preset high pitch are used to comprehensively limit the processing method of performing melody transformation processing on the melody track.

[0122] In some embodiments, the note pitches corresponding to multiple music notes in the melody track are compared with a preset low pitch, and the note pitches corresponding to multiple music notes are compared with a preset high pitch to determine the bass note ratio and the treble note ratio.

[0123] Among them, the bass note ratio is used to indicate the note ratio of bass notes to multiple music notes, and bass notes are used to indicate music notes that are lower than the preset bass pitch; the treble note ratio is used to indicate the note ratio of treble notes to multiple music notes, and treble notes are used to indicate music notes that are higher than the preset treble pitch.

[0124] In some embodiments, a melody transformation process is performed on the melody track based on the bass note ratio and the treble note ratio to obtain an updated melody track as melody data information.

[0125] In an illustrative manner, after obtaining the melody track, the note pitches corresponding to the multiple music notes are determined; a preset low pitch and a preset high pitch are obtained, and the multiple music notes are compared with the preset low pitch and the preset high pitch respectively to calculate the bass note ratio and the treble note ratio respectively.

[0126] For example: taking the preset bass pitch as 56 and the preset treble pitch as 70 as an example, after determining the note pitches corresponding to multiple music notes, the multiple note pitches are compared with 56 and 70 respectively. If the note pitch corresponding to the music note is lower than 56, the music note is regarded as a bass note; if the note pitch corresponding to the music note is higher than 70, the music note is regarded as a treble note; thereby, the first note number of the bass note and the second note number of the treble note can be determined, and then the ratio of the bass notes is determined by combining the number of the first notes and the number of music notes, and the ratio of the treble notes is determined by combining the number of the second notes and the number of music notes.

[0127] Indicatively, when both the bass note ratio and the treble note ratio meet the preset conditions, the melody track is not adjusted.

[0128] For example: when the proportion of high notes exceeds 10% and the proportion of bass notes is less than 20%, the melody track will not be processed in the higher octave; when the above preset conditions are not met, the melody track will be adjusted.

[0129] In some embodiments, to ensure a sense of completeness and closure, an outro can be added to each piece of music, ranging in length from one to three bars. Since different pieces of music have different ending bars, some end with a single, long note, some with a long pause, and some without enough time for an outro, it's necessary to use the melody track's information to determine in advance whether to add an outro to the accompaniment track.

[0130] Indicatively, when the last note of the ending measure of the melody track ends after the second beat and is not a long note, a blank ending measure is pre-added when generating the accompaniment track, and the chords are copied from the previous measure; thus, when performing texture conversion on the ending, if the selected texture exceeds four beats, it will be extended to two or three measures when writing to MIDI.

[0131] Optionally, melody data information and chord data information are obtained based on the melody track and the chord track through the above process, and the melody data information and chord data information are used to express the music data in digital form by integrating the beat data information.

[0132] It should be noted that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0133] To sum up, in the process of generating music accompaniment data through music data, the music beat and music notes are quantized through the music information extraction process, which improves the accuracy of the analysis of the music data and makes the accompaniment data generation process more targeted; then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, and then the music accompaniment that can more accurately present the music data is obtained through the music accompaniment data rendering, thereby improving the generation stability and generation effect of the music accompaniment.

[0134] In an optional embodiment, the accompaniment textures corresponding to the multiple music bars in the music data are first obtained, and then the corresponding music bars are adjusted according to the accompaniment textures, so as to generate the music accompaniment data by combining the multiple adjusted music bars. Figure 4 As shown above Figure 2 The illustrated step 230 may also be implemented as the following steps 410 to 430 .

[0135] Step 410 , in the case of using music bars as units, using the melody data information as a generation constraint for the music accompaniment, and determining accompaniment textures corresponding to a plurality of music bars based on the beat data information and the chord data information.

[0136] Among them, a music measure, also known as a bar, includes a preset number of music beats and is a basic organizational unit for detailed analysis of music data, used to divide and organize music clips; introducing bars to analyze music data facilitates improving the understanding of music measures.

[0137] The unit of music bar is used to indicate that each music bar of the music data is analyzed separately. For each music bar, the melody data information is used as a generation constraint condition for the music accompaniment, thereby determining the accompaniment texture based on the beat data information and the chord data information.

[0138] Among them, the accompaniment texture is a sound element used to set off the main melody corresponding to the melody data information. Schematically, the accompaniment texture refers to the sound layer in the music that is responsible for supporting and setting off the main melody. This layer usually includes chords, bass, drums, percussion and other elements. Their combination forms the foundation and background of the music. The role of the accompaniment texture in music is to provide support for the main melody and add layers and richness to the music. The rational use of accompaniment texture can make the music more complete and rich, and enhance the audience's musical experience. Different types of music may have different ways of constructing accompaniment textures to suit different styles and emotional expressions.

[0139] In some embodiments, accompaniment textures corresponding to a plurality of music measures are determined, so that music notes in the music measures are adjusted by the accompaniment textures, thereby enriching the expression of the music measures.

[0140] Optionally, when determining the accompaniment texture, the melody, beat and chords are comprehensively adjusted to determine the accompaniment texture that is conducive to improving the musical performance of the music section.

[0141] In an optional embodiment, the texture determination rule is obtained with music bars as units.

[0142] The texture determination rule is used to determine the accompaniment texture corresponding to the music measure through the music information extraction result.

[0143] Illustratively, the music information extraction result is used to represent the result obtained through the music information extraction process, and the music information extraction result includes melody data information, beat data information, and chord data information.

[0144] That is, the obtained texture determination rule is used to integrate the melody data information, beat data information and chord data information in the music measure to determine the accompaniment texture corresponding to the music measure.

[0145] In an optional embodiment, under the limitation of the texture determination rules, the note change amplitude of the music accompaniment is limited by the melody data information, the rhythm change of the music accompaniment is limited by the beat data information, and the degree of coordination of the notes of the music accompaniment is limited by the chord data information, so as to determine the accompaniment textures corresponding to the music measures respectively.

[0146] Illustratively, the melody data information is closely related to the music notes, so the melody data information can be used to assist in determining the pitch, changes, etc. of the accompaniment notes in the music accompaniment, that is, the melody data information can be used to limit the note change amplitude of the music accompaniment.

[0147] Optionally, the melody data information is used to limit the note variation amplitude corresponding to the music accompaniment through the pitch range, pitch density and the pitch value of the lowest note.

[0148] Schematically, the beat data information is closely related to the beat of the music composed of the music notes, so the beat data information can be used to assist in determining the note rhythm, note change rate, etc. of the accompaniment notes in the music accompaniment.

[0149] Optionally, the beat data information is used to determine the rhythm type corresponding to the music accompaniment according to the number of music beats per unit time, and the rhythm types include slow, medium and fast.

[0150] Schematically, the chord data information, as information determined by the chord content composed of at least two notes, is closely related to the music notes themselves. Therefore, the chord data information can be used to assist in determining the note form of the accompaniment notes in the music accompaniment, that is, to determine the notes that should be used in the music accompaniment, thereby more meticulously improving the accuracy of the music accompaniment.

[0151] Optionally, the chord data information is used to assist the melody data information in determining the degree of note harmony through chord positions.

[0152] In some embodiments, each music measure is analyzed to obtain the accompaniment texture corresponding to each music measure.

[0153] Alternatively, the classification of accompaniment textures is often based on the combination of different parts in a piece of music (e.g., harmony, bass, percussion, etc.) and their role in the piece. Accompaniment textures include: harmonic accompaniment (based on chords, responsible for supporting the main melody), rhythmic accompaniment (based on drums and percussion, responsible for providing rhythmic sense), bass accompaniment (based on bass instruments, providing a stable bass foundation for the music), and melodic accompaniment (accompanied by the main melody, highlighting the main melody through harmony, timbre, or musical ornamentation).

[0154] That is: based on the melody, chords and beats, the accompaniment texture type corresponding to each music measure is comprehensively determined, so that the corresponding music measure can be adjusted through the accompaniment texture.

[0155] In some embodiments, when performing texture conversion on a bar-by-bar basis, each song is processed in a loop from the beginning to the end. To accommodate songs of varying tempos, three texture conversion routines are designed based on BPM. A BPM of 90 or less is considered slow, a BPM of 90 or more but less than 120 is considered medium, and a BPM of 120 or more is considered fast. Slow songs have the richest variety of rhythmic textures, while medium-tempo songs often have a minimum note duration of eighth notes, while fast songs have a minimum note duration of quarter notes.

[0156] After the speed judgment is completed, enter the melody judgment of the measure. Calculate the pitch range, note density and the pitch of the lowest note of the melody, and select a high or low, sparse or dense accompaniment texture. In each case, the selection of accompaniment texture from the alternatives that meet the conditions also has a certain degree of randomness. For the first measure, it is necessary to judge whether it meets the requirements of a weak-start measure and whether to add a prelude. The judgment rules are: non-weak-start measures or the first note starts before the second beat are treated as normal measures. Weak-start measures but the first note appears before the third beat are not accompanied by accompaniment. Weak-start measures but the first note appears on the third beat and later are randomly selected to have no accompaniment or a prelude. The prelude only selects the accompaniment texture based on the first chord.

[0157] After selecting the accompaniment texture, to avoid errors in the total duration of the notes in the measure, which could lead to cumulative deviations, you can set a limit on the number of beats per measure: within a limited number of cycles, the sum of the durations of all the notes in the current measure is checked to see if it equals 4 beats. If it exceeds, the last note or rest is removed; if it falls short, a rest is added. To avoid disrupting the designed accompaniment texture, measures that require an introductory or ending measure are exempt from this limit.

[0158] For the last accompaniment measure, you can first determine the melody: for an empty measure, play the coda directly; if the rest reaches three beats and the last note ends on or before the first beat, or the note is dragged out, add a one-beat chord and then play the coda; for the rest, add two-beat chords and then play the coda.

[0159] It should be noted that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0160] Step 420 , adjusting the duration of the music notes in the corresponding music measure by using the accompaniment texture to obtain adjusted measures corresponding to the plurality of music measures.

[0161] Illustratively, after obtaining the accompaniment texture corresponding to each music measure, the music notes in the music measure can be adjusted through the accompaniment texture.

[0162] Optionally, the note duration corresponding to the music note is adjusted through the accompaniment texture, and the note duration is used to indicate the duration of the music note.

[0163] For example, the duration of a musical note can be modified within a measure to make it shorter or longer. This can be achieved by splitting the musical note into shorter notes or combining multiple musical notes into a single longer note. For example, a quarter note can be split into two eighth notes, or two eighth notes can be combined into a quarter note.

[0164] In addition, when modifying the duration of musical notes, you can also use legato or staccato to adjust the note values. That is, legato connects adjacent musical notes to make them closer in time; staccato creates a short interval between musical notes to add a sense of space.

[0165] Optionally, you can also add ornaments to the music notes in the music measure through the accompaniment texture, such as trills, glissando, legato, etc., which can make the original music notes more expressive. These ornaments can increase the changes in the music without changing the basic melody.

[0166] Optionally, new rhythmic patterns can be introduced into the music measure through the accompaniment texture, for example: introducing more complex drum beats, emphasis on weak beats or cross-rhythms. This process can be achieved by making changes to the drum beats or other percussion instruments in the accompaniment texture.

[0167] Optionally, the accompaniment texture can be used to emphasize weak beats or other parts of the beat in the music measure to make the music accompaniment more dynamic.

[0168] Optionally, the accompaniment texture can be accelerated or decelerated at the end of the musical measure, thereby creating a gradual change, etc.

[0169] Illustratively, the above content is determined based on the accompaniment texture corresponding to the music measure, and the accompaniment texture is determined comprehensively with the help of texture determination rules, melody, chords and beats.

[0170] The adjustment measure includes a plurality of adjusted music notes.

[0171] Schematically, the music notes in the corresponding music measure are adjusted through the accompaniment texture to obtain multiple adjusted music notes, thereby determining the adjusted music notes in each music measure, and each adjusted music measure is called an adjusted measure.

[0172] Step 430: Generate music accompaniment data by combining multiple adjustment measures.

[0173] Illustratively, the temporal relationship between a plurality of music measures in the music data is determined; and the plurality of adjustment measures are combined according to the temporal relationship, thereby generating music accompaniment data.

[0174] Alternatively, since the adjustment measure is obtained by adjusting the musical notes through the accompaniment texture, the adjustment measure focuses on expressing the digital information of the musical accompaniment, and the musical accompaniment data generated by the plurality of adjustment measures also focuses on expressing the digital information of the musical accompaniment. For example, the musical accompaniment data is information expressed in the form of a MIDI file.

[0175] It is worth noting that the above distances are merely illustrative and are not limited in the embodiments of the present application.

[0176] To sum up, in the process of generating music accompaniment data through music data, the music beat and music notes are quantized through the music information extraction process, which improves the accuracy of the analysis of the music data and makes the accompaniment data generation process more targeted; then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, and then the music accompaniment that can more accurately present the music data is obtained through the music accompaniment data rendering, thereby improving the generation stability and generation effect of the music accompaniment.

[0177] In an optional embodiment, when rendering audio data using music accompaniment data, in order to improve the melody accuracy of the music accompaniment data, the melody data information and the music accompaniment data can be mixed, and then the music result can be obtained by rendering the audio data based on the mixed result. Figure 4 As shown above Figure 2 The embodiment shown can also be implemented as the following steps 510 to 550; Figure 2 The illustrated step 240 may also be implemented as the following steps 540 to 550 .

[0178] Step 510: Acquire music data.

[0179] The music data is divided into a plurality of music beats, and the music beats include music notes.

[0180] Schematically, step 510 has been described in the above step 210 and will not be repeated here.

[0181] Step 520: extract music information from the music data to obtain beat data information, chord data information, and melody data information.

[0182] Among them, the beat data information is used to represent the changing speed of multiple music beats, the chord data information is used to describe the chord units extracted based on the music beat, and the melody data information is used to describe the note changes between multiple music notes.

[0183] Illustratively, step 520 has been described in the above step 220 and steps 310 to 330 and will not be repeated here.

[0184] Step 530: Using the melody data information as a generation constraint for the music accompaniment, the music accompaniment data is generated through the beat data information and the chord data information.

[0185] The melody data information is used to define the accompaniment melody of the music accompaniment with the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data through music parameters.

[0186] Illustratively, step 530 has been described in the above step 230 and steps 410 to 430 and will not be repeated here.

[0187] Step 540: Mix the melody data information and the music accompaniment data to obtain a mixing result.

[0188] In an illustrative manner, in order to more appropriately present the music melody while reflecting the accompaniment, the melody data information corresponding to the music melody is mixed with the generated music accompaniment data.

[0189] Mixing refers to the process of combining different audio tracks (such as melody and accompaniment) to create a complete musical composition. The purpose of mixing is to combine the individual tracks into a complete musical work while maintaining audio clarity and balance. The purpose of mixing melody data and music accompaniment data is to create a sense of unity in the music, so that the music melody data corresponding to the music data itself and the generated music accompaniment data form an organic whole, so that after mixing, they can complement each other without suppressing each other, forming balanced, compressed, and reverberant audio effects.

[0190] In an optional embodiment, the melody data information and the music accompaniment data are mixed by a sound source reverberator to generate a mixing result.

[0191] In an exemplary embodiment, the melody data information and the music accompaniment data are both implemented as MIDI files, and the melody data information and the music accompaniment data involved in the mixing process are combined to create a complete MIDI file. This can be adjusted in terms of notes, volume, timbre, etc. to create a more complex and rich musical work.

[0192] In some embodiments, the music melody track corresponding to the melody data information and the music accompaniment track corresponding to the music accompaniment data are determined; under the condition that the audio tracks are aligned, the music melody track and the music accompaniment track are mixed to generate a mixing result.

[0193] Illustratively, the melody data represents the melody track of a music piece, including the representation of the music notes based on the melody data; the accompaniment data represents the accompaniment track of the music piece, including the representation of the music notes based on the accompaniment data. Audio track alignment is used to mix the melody track and accompaniment track of the music piece, while aligning the two tracks to generate a mixed audio result.

[0194] Optionally, the audio mixing process may be implemented as follows.

[0195] (1) Import melody data and music accompaniment data: Import the two pieces of information that need to be mixed into MIDI editing software or a digital audio workstation (DAW), and ensure that they are correctly aligned on the timeline.

[0196] (2) Adjust the starting point: Make sure that the melody data information and the music accompaniment data start from the same starting point on the time axis so that they can be synchronized.

[0197] (3) Note Editing: Edit the notes in the melody data and music accompaniment data. For example, you can delete or adjust some notes to make them blend better. You can also consider adjusting the duration of some notes to create richer chords or melody changes.

[0198] (4) Volume balance: Adjust the volume of the melody data information and the music accompaniment data to ensure that they can maintain a balance after mixing. This can be achieved by adjusting the volume controllers on the channels corresponding to the melody data information and the music accompaniment data respectively.

[0199] (5) Tone and expression: During the mixing process, the tone (instrument selection) and expression controller (such as volume, note length, tone, etc.) on the channels corresponding to the melody data information and music accompaniment data can be adjusted to make the overall music more expressive.

[0200] (6) Rhythm and time sense: Adjust the rhythm and time sense of the melody data information and the music accompaniment data so that they are better coordinated after mixing, such as: time offset of music notes, adding some artificial micro-time value changes, etc.

[0201] (7) Effect processing: Apply some transformation effects, such as merge effect, delay effect, chorus effect, etc., to increase the complexity and layering of the mix.

[0202] (8) Mixing and exporting: In the final stage of the mixing process, after ensuring that the mixing effect of the melody data information and the music accompaniment data meets the preset requirements, the mixed results are exported and the mixing results are realized in the form of MIDI files.

[0203] Step 550 , performing audio data rendering on the mixing result to obtain a music result corresponding to the music data.

[0204] Among them, the music result uses the music accompaniment as the accompaniment content.

[0205] In an illustrative manner, after obtaining the mixing result, the audio data of the mixing result is rendered to obtain a music result that expresses the complete music content. Since the generation of the music result depends on the music accompaniment data, the music result includes the music accompaniment corresponding to the music accompaniment data, which is conducive to more realistically presenting the music content through melody data information while generating the music accompaniment.

[0206] Alternatively, the mixing data is implemented as a MIDI file, a digital representation of the music data. To obtain the resulting music piece corresponding to the mixing data, the MIDI file contains musical information such as notes, pitches, and durations, but does not contain sound waveforms. Therefore, the MIDI file must be converted into data for rendering the resulting music piece. Specifically, audio conversion is first performed based on the mixing result to obtain the resulting music piece audio; then, audio rendering is performed on the resulting music piece audio to obtain the resulting music piece.

[0207] Optionally, the music result audio after the audio conversion is rendered to obtain the music result, for example, the music result audio is opened through a terminal to play the music result with music accompaniment.

[0208] It should be noted that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0209] To sum up, in the process of generating music accompaniment data through music data, the music beat and music notes are quantized through the music information extraction process, which improves the accuracy of the analysis of the music data and makes the accompaniment data generation process more targeted; then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, and then the music accompaniment that can more accurately present the music data is obtained through the music accompaniment data rendering, thereby improving the generation stability and generation effect of the music accompaniment.

[0210] In an optional embodiment, the music data can be implemented as any popular song, and the method for generating music accompaniment is applied to the accompaniment generation process of popular music. The method can also be called "a method for automatically generating piano covers of popular songs in the symbol domain", such as Figure 6 As shown, this is an overall framework diagram of an embodiment of the present application.

[0211] Illustratively, after obtaining the original song audio file 610 (i.e., the above-mentioned music data), a preset algorithm (such as a neural network transform) is first used to perform melody extraction, beat tracking, and chord extraction on the original song audio file 610 to obtain music information, including melody notes 621 after melody extraction, beat information 622 after beat tracking, and chord sequence 623 after chord extraction.

[0212] The extracted melody note 621 is represented by a starting point, an offset point and a pitch; the melody note 621 is aligned with the beat information 622 detected from the original song audio file 610 and quantized to sixteenth notes to obtain a melody MIDI file (i.e., the above-mentioned melody data information).

[0213] In chord extraction, one beat is used as the smallest unit, including major and minor triads, and a chord sequence 623 is obtained.

[0214] The melody and chord sequence are combined into a MIDI file after deleting the bars without melody, with the time signature being 4 / 4.

[0215] Furthermore, the tempo information (BPM) of the original audio file 610 is estimated from the beat information 622, and the average tempo of the original audio file 610 is slightly slowed down. Secondly, while controlling the accompaniment's range and density based on the melody, the accompaniment generation algorithm generates a texture that matches the melody based on the chord progression and type, and outputs a piano accompaniment MIDI file (i.e., music accompaniment data). Finally, the two MIDI tracks of melody and accompaniment are rendered into a piano cover audio file (i.e., the music audio used to render the music accompaniment).

[0216] In some embodiments, the input of the accompaniment generation algorithm is as follows: Figure 7 As shown, the input file includes multiple notes 710 in the ".mid" format, i.e., a MIDI file. It stores two tracks, a melody track and a chord track, including time signature (both in 4 / 4) and key signature metadata. The chords are pre-processed to specific pitches rather than chord notations such as "C" or "Am." The input chord pitch range is within the default pitch range (starting from the first octave), considering only major and minor triads in their original position. However, the texture includes inversions, sevenths, and ninths.

[0217] The accompaniment generation method is performed with the help of an accompaniment generation algorithm, such as Figure 8 As shown, it mainly includes the following five processes: (1) input reading 810: input MIDI and BPM reading; (2) preprocessing 820: chord track preprocessing; (3) texture conversion 830; (4) post-processing 840: melody and accompaniment post-processing; (5) writing MIDI file 850.

[0218] (1) Input read 810

[0219] Schematically, the input MIDI file is read, the melody and chords are divided into two tracks and stored in two parts, and the estimated BPM of the original song is input as one of the conditions for selecting the accompaniment texture.

[0220] (2) Preprocessing 820

[0221] The number of bars in the melody and chord tracks is calculated separately. If the number of bars in the chord track is less than the number of bars in the melody track, the number of bars is filled in, and the chords of the previous bar are copied to the added bar. In addition, each bar is checked. To avoid long rests in the accompaniment, if a bar does not have any chords, the algorithm copies the chords of the previous bar. If the bar is the first bar, the algorithm copies the chords of the first subsequent bar that is not a full rest.

[0222] For songs with generally low-pitched melodies, the melody can be raised an octave to improve the listening experience and prevent conflict with the accompaniment. The specific determination method is to use MIDI pitch 56 and 70 as thresholds, calculate the proportion of bass notes and treble notes respectively, and if the treble note ratio exceeds 10% and the bass note ratio is less than 20%, the octave raising is not performed.

[0223] In order to make the music feel complete and final, an ending can be added to each piece of music, with a length ranging from one measure to three measures. Since the ending measures of the melody of different songs are different, some end with a long note, some end with a long rest, and some do not have enough time to add an ending. Therefore, it is necessary to determine in advance whether to add an ending measure to the accompaniment track. The specific judgment rule is: when the last note of the melody ending measure ends after the second beat and is not a long note, a blank ending measure is added in advance, and the chords copy the chords of the previous measure. When converting the texture of the ending, if the selected texture exceeds four beats, it will be extended to two to three measures when writing to MIDI.

[0224] (3) Texture conversion 830

[0225] Texture conversion is performed in measure units, looping through each song from beginning to end. To accommodate songs of varying tempos, three texture conversion routines have been designed based on BPM. BPMs up to 90 are considered slow, BPMs above 90 but below 120 are considered medium, and BPMs above 120 are considered fast. Slow songs have the richest variety of rhythmic textures, while medium-tempo songs typically have a minimum note duration of eighth notes, and fast songs have a minimum note duration of quarter notes.

[0226] After the speed judgment is completed, enter the melody judgment of the measure. Calculate the pitch range, note density and the pitch of the lowest note of the melody, and select a high or low, sparse or dense accompaniment texture. In each case, the selection of texture from the alternatives that meet the conditions also has a certain degree of randomness. For the first measure, it is necessary to determine whether it meets the requirements of a weak-start measure and whether to add a prelude. The specific rules are: non-weak-start measures or the first note starts before the second beat are treated as normal measures. Weak-start measures but the first note appears before the third beat are not accompanied by accompaniment. Weak-start measures but the first note appears on the third beat and later are randomly selected without accompaniment or with a prelude. The prelude only selects the texture based on the first chord.

[0227] After selecting a texture, a beat limit is set to prevent errors in the total duration of the notes in the measure, which could lead to cumulative deviations. Within a limited number of iterations, the system checks whether the sum of the durations of all the notes in the measure equals four beats. If it exceeds, the last note or rest is removed; if it falls short, a rest is added. To avoid disrupting the designed texture, measures requiring an introductory or coda are exempt from this limit.

[0228] For the last accompaniment measure, the algorithm first determines the melody: if the measure is empty, the coda is played directly; if the rest reaches three beats and the last note ends on or before the first beat, or the note is prolonged, a one-beat chord is added and then a coda is played; for the rest, a two-beat chord is added and then a coda is played.

[0229] In some embodiments, the texture types include column, semi-decomposed, and fully decomposed, and the rhythmic patterns include quarter, twenty-eighth, sixteenth, syncopated, dotted, quintuplets, and other types, and include ornaments such as arpeggios, which are rich in diversity. When the input only has major and minor triads, sevenths, ninths, thirteenths, and chord inversions are selectively added according to the range and density of the melody to expand the chord types. Taking into account the speed of the song, various specially designed textures can cover songs of various styles, such as lyrical slow songs, medium-speed songs, and fast and popular songs, while maintaining a soothing overall style. In addition, the prelude texture is designed for the weak start bar with a rest of three beats, and is randomly selected for the two situations of no accompaniment for the weak start bar. In order to give the music a sense of ending, a variety of codas are also designed, which are selected based on the note value, end position, and other information of the ending bar of the melody.

[0230] Texture conversion is divided into two conversion methods: function and file. Function conversion is simple and stable and can be applied to all input chord types, while file conversion is more conducive to expansion. Function conversion refers to the use of relevant texture conversion functions to target various chord durations within a measure, directly select basic notes from the chords, and design pitches and durations based on harmony, combine single notes and chords, generate complete texture measures and return them. File conversion refers to the use of digital music score files, that is, ".mxl" files to store the textures of each measure, according to the two cases of major triads and minor triads, and the four cases of chord durations of four beats, three beats, two beats, and one beat. Each type of texture is divided into eight files. Each measure in the score file stores a texture, which are all C or Cm chords and their variations. When used, it is transposed according to the interval relationship between the current chord root and the C4 pitch. For example Figure 9 A plurality of music notes 910 are shown, and the plurality of music notes 910 represent a Cadd9 chord, which is an example of a texture file with a duration of 4 beats.

[0231] (4) Post-processing 840

[0232] After completing the texture conversion, the algorithm first inserts the estimated BPM of the original song into the current score. Except for the first and last measures, the accompaniment is pitch-adjusted for optimal listening: single notes below G2 are pitched an octave higher, and chord notes below A2 are pitched an octave lower. Accompaniment notes higher than the melody are pitched an octave lower. Next, the algorithm compares notes with similar starting points in the melody and accompaniment, adjusting for dissonant major and minor seconds and identical notes. Furthermore, repeated identical notes in the accompaniment are legato (excluding chords). Finally, the dynamics of the melody and accompaniment are adjusted for later audio production.

[0233] (5) Write MIDI file 850

[0234] After completing all the above steps, save the melody and accompaniment as two MIDI files respectively to facilitate batch rendering of audio later.

[0235] It should be noted that the above are merely illustrative examples and are not limited to the embodiments of the present application.

[0236] To sum up, in the process of generating music accompaniment data through music data, the music beat and music notes are quantized through the music information extraction process, which improves the accuracy of the analysis of the music data and makes the accompaniment data generation process more targeted; then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, and then the music accompaniment that can more accurately present the music data is obtained through the music accompaniment data rendering, thereby improving the generation stability and generation effect of the music accompaniment.

[0237] Figure 10 This is a structural block diagram of a device for generating music accompaniment provided by an exemplary embodiment of the present application. Figure 10 As shown, the device includes the following parts:

[0238] The data acquisition module 1010 is used to acquire music data, where the music data is divided into a plurality of music beats, and the music beats include music notes;

[0239] an information extraction module 1020 for extracting music information from the music data to obtain beat data information, chord data information, and melody data information, wherein the beat data information is used to characterize the speed of change of the multiple music beats, the chord data information is used to describe the chord units extracted based on the music beat, and the melody data information is used to describe the note changes between the multiple music notes;

[0240] a data generation module 1030 configured to use the melody data information as a generation restriction condition for the music accompaniment and generate music accompaniment data using the beat data information and the chord data information, wherein the melody data information is used to restrict the accompaniment melody of the music accompaniment using the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data using music parameters;

[0241] The accompaniment generation module 1040 is configured to render audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

[0242] In an optional embodiment, the information extraction module 1020 is further used to determine the beat data information corresponding to the music data based on the number of the multiple music beats in unit time; perform music information extraction on the music data to determine the chord track and melody track corresponding to the music data, the chord track is used to describe the sound group constructed by the multiple music notes in the form of chords, and the melody track is used to describe the main melody composed of the multiple music notes according to the time sequence relationship; based on the melody track and the chord track, the melody data information and the chord data information corresponding to the music data are obtained respectively.

[0243] In an optional embodiment, the information extraction module 1020 is further used to respectively determine the number of bars in the melody track and the number of bars in the chord track, and obtain a first number corresponding to the melody track and a second number corresponding to the chord track, wherein the first number is used to represent the number of bars in the melody track, and the second number is used to represent the number of bars in the chord track; based on the first number, the second number of the chord track is adjusted to obtain the adjusted chord track as the chord data information; based on the note pitches corresponding to the multiple music notes in the melody track, the melody track is subjected to melody transformation processing to obtain an updated melody track as the melody data information.

[0244] In an optional embodiment, the information extraction module 1020 is further used to obtain a preset low pitch and a preset high pitch, and the preset low pitch and the preset high pitch are used to comprehensively limit the processing method for performing melody transformation processing on the melody track; the note pitches corresponding to the multiple music notes in the melody track are compared with the preset low pitch, and the note pitches corresponding to the multiple music notes are compared with the preset high pitch to determine the bass note ratio and the treble note ratio; the bass note ratio is used to indicate the note ratio of bass notes to the multiple music notes, and the bass notes are used to indicate music notes lower than the preset low pitch; the treble note ratio is used to indicate the note ratio of treble notes to the multiple music notes, and the treble notes are used to indicate music notes higher than the preset high pitch; the melody transformation processing is performed on the melody track based on the bass note ratio and the treble note ratio to obtain the updated melody track as the melody data information.

[0245] In an optional embodiment, the data generation module 1030 is further used to determine, in units of music bars, the melody data information as a generation restriction condition for the music accompaniment, and based on the beat data information and the chord data information, accompaniment textures corresponding to multiple music bars respectively; wherein a music bar includes a preset number of music beats, and the accompaniment texture is a sound element used to set off the main melody corresponding to the melody data information; the note duration of the music notes in the corresponding music bar is adjusted by the accompaniment texture to obtain adjusted bars corresponding to multiple music bars respectively, the note duration is used to indicate the duration of the music notes, and the adjusted bar includes multiple adjusted music notes; and the music accompaniment data is generated by combining multiple adjusted bars.

[0246] In an optional embodiment, the data generation module 1030 is also used to obtain a texture determination rule based on the music bar, and the texture determination rule is used to determine the accompaniment texture corresponding to the music bar through the music information extraction result, and the music information extraction result includes the beat data information, the chord data information and the melody data information; under the limitation of the texture determination rule, the note change amplitude of the music accompaniment is limited by the melody data information, the rhythm change of the music accompaniment is limited by the beat data information, and the note coordination degree of the music accompaniment is limited by the chord data information, so as to determine the accompaniment textures corresponding to the multiple music bars respectively.

[0247] In an optional embodiment, the melody data information is used to limit the note variation amplitude corresponding to the music accompaniment through pitch range, pitch density and the pitch value of the lowest note;

[0248] The beat data information is used to determine the rhythm type corresponding to the music accompaniment according to the number of music beats per unit time, and the rhythm type includes slow, medium and fast;

[0249] The chord data information is used to assist the melody data information in determining the harmony of the notes through chord positions.

[0250] In an optional embodiment, the accompaniment generation module 1040 is further used to mix the melody data information and the music accompaniment data to obtain a mixing result; perform audio data rendering on the mixing result to obtain a music result corresponding to the music data, wherein the music result uses the music accompaniment as the accompaniment content.

[0251] In an optional embodiment, the accompaniment generation module 1040 is further used to determine the music melody track corresponding to the melody data information, and the music accompaniment track corresponding to the music accompaniment data; under the condition that the audio tracks are aligned, the music melody track and the music accompaniment track are mixed to generate the mixing result.

[0252] To sum up, in the process of generating music accompaniment data through music data, the music beat and music notes are quantized through the music information extraction process, which improves the accuracy of the analysis of the music data and makes the accompaniment data generation process more targeted; then, the melody data information, beat data information and chord data information are integrated, and the beat and chords are incorporated into the accompaniment melody generation process under the limitation of the music melody, so that the music accompaniment data can be presented more carefully through music parameters, and then the music accompaniment that can more accurately present the music data is obtained through the music accompaniment data rendering, thereby improving the generation stability and generation effect of the music accompaniment.

[0253] It should be noted that the above-described embodiment of the apparatus for generating a musical accompaniment is merely illustrative of the division of the aforementioned functional modules. In actual applications, the aforementioned functions can be assigned to different functional modules as needed, i.e., the internal structure of the device can be divided into different functional modules to perform all or part of the functions described above. Furthermore, the apparatus for generating a musical accompaniment and the embodiment of the method for generating a musical accompaniment are conceptually identical. The specific implementation process is detailed in the method embodiment and will not be further elaborated here.

[0254] Figure 11 The following is a schematic diagram of the structure of a server provided by an exemplary embodiment of the present application. The server 1100 includes a central processing unit (CPU) 1101, a system memory 1104 including a random access memory (RAM) 1102 and a read-only memory (ROM) 1103, and a system bus 1105 connecting the system memory 1104 and the CPU 1101. The server 1100 also includes a mass storage device 1106 for storing an operating system 1113, application programs 1114, and other program modules 1115.

[0255] The mass storage device 1106 is connected to the central processing unit 1101 through a mass storage controller (not shown) connected to the system bus 1105. The mass storage device 1106 and its associated computer-readable media provide non-volatile storage for the server 1100. In other words, the mass storage device 1106 may include a computer-readable medium (not shown) such as a hard disk or a compact disc read-only memory (CD-ROM) drive.

[0256] Without further generality, computer-readable media may include computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. The system memory 1104 and mass storage device 1106 described above may be collectively referred to as memory.

[0257] According to various embodiments of the present application, the server 1100 may also be connected to a remote computer on a network such as the Internet for operation. That is, the server 1100 may be connected to the network 1112 via the network interface unit 1111 connected to the system bus 1105, or the network interface unit 1111 may be used to connect to other types of networks or remote computer systems (not shown).

[0258] The memory also includes one or more programs, which are stored in the memory and configured to be executed by the CPU.

[0259] An embodiment of the present application also provides a computer device, which includes a processor and a memory, wherein the memory stores at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by the processor to implement the method for generating musical accompaniment provided by the above-mentioned method embodiments.

[0260] An embodiment of the present application also provides a computer-readable storage medium, on which is stored at least one instruction, at least one program, code set or instruction set, and the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the method for generating musical accompaniment provided by the above-mentioned method embodiments.

[0261] Embodiments of the present application further provide a computer program product or computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method for generating a musical accompaniment described in any of the above embodiments.

[0262] The above description is merely an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A method for generating a music accompaniment, characterized in that: The method comprises: Acquire music data, where the music data is divided into a plurality of music beats, and the music beats include music notes; Determining beat data information corresponding to the music data based on the number of the multiple music beats within a unit time, wherein the beat data information is used to represent a change speed of the multiple music beats; Extracting music information from the music data to determine a chord track and a melody track corresponding to the music data, wherein the chord track is used to describe a sound group constructed by the multiple music notes in the form of chords, and the melody track is used to describe a main melody constructed by the multiple music notes in a time sequence relationship; Determining the number of bars in the melody track and the number of bars in the chord track respectively, obtaining a first number corresponding to the melody track and a second number corresponding to the chord track, wherein the first number is used to represent the number of bars in the melody track and the second number is used to represent the number of bars in the chord track; adjusting the second number of the chord track based on the first number to obtain the adjusted chord track as chord data information, wherein the chord data information is used to describe chord units extracted based on the beat of the music; performing a melody transformation process on the melody track based on the pitches of the notes corresponding to the plurality of music notes in the melody track, thereby obtaining an updated melody track as melody data information, wherein the melody data information is used to describe the note changes between the plurality of music notes; The melody data information is used as a generation restriction condition for the music accompaniment, and music accompaniment data is generated by using the beat data information and the chord data information, wherein the melody data information is used to limit the accompaniment melody of the music accompaniment by the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data by using music parameters; Audio data rendering is performed based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

2. The method according to claim 1, characterized in that The step of performing melody transformation processing on the melody track based on the pitches of the notes respectively corresponding to the plurality of music notes in the melody track to obtain an updated melody track as melody data information includes: Obtaining a preset low pitch and a preset high pitch, wherein the preset low pitch and the preset high pitch are used to comprehensively define a processing method for performing melody transformation processing on the melody track; Comparing the note pitches corresponding to the plurality of music notes in the melody track with the preset low pitch, and comparing the note pitches corresponding to the plurality of music notes with the preset high pitch, to determine a bass note ratio and a treble note ratio; the bass note ratio is used to indicate the note ratio of bass notes to the plurality of music notes, and the bass notes are used to indicate music notes lower than the preset low pitch; the treble note ratio is used to indicate the note ratio of treble notes to the plurality of music notes, and the treble notes are used to indicate music notes higher than the preset high pitch; A melody transformation process is performed on the melody track based on the bass note ratio and the treble note ratio to obtain an updated melody track as the melody data information.

3. The method according to any one of claims 1 to 2, characterized in that: The step of using the melody data information as a generation restriction condition for the music accompaniment and generating the music accompaniment data through the beat data information and the chord data information includes: In a case where a musical measure is used as a unit, the melody data information is used as a constraint condition for generating the musical accompaniment, and based on the beat data information and the chord data information, accompaniment textures corresponding to the plurality of musical measures are determined; wherein a musical measure includes a preset number of musical beats, and the accompaniment textures are sound elements used to set off the main melody corresponding to the melody data information; Adjusting the note durations of the music notes in the corresponding music measure using the accompaniment texture to obtain adjusted measures corresponding to the plurality of music measures, wherein the note durations are used to represent the durations of the music notes, and the adjusted measures include the plurality of adjusted music notes; The music accompaniment data is generated in combination of a plurality of adjustment measures.

4. The method according to claim 3, characterized in that The method of determining accompaniment textures corresponding to a plurality of music bars based on the beat data information and the chord data information, using the melody data information as a generation restriction condition for the music accompaniment, in a case where the music bar is used as a unit, includes: In the case of using the music bar as a unit, obtaining a texture determination rule, the texture determination rule being used to determine the accompaniment texture corresponding to the music bar according to a music information extraction result, the music information extraction result including the beat data information, the chord data information, and the melody data information; Under the limitation of the texture determination rules, the note change amplitude of the music accompaniment is limited by the melody data information, the rhythm change of the music accompaniment is limited by the beat data information, and the degree of note coordination of the music accompaniment is limited by the chord data information, so as to determine the accompaniment textures corresponding to the multiple music measures respectively.

5. The method according to any one of claims 1 to 2, characterized in that: The melody data information is used to limit the note variation amplitude corresponding to the music accompaniment through pitch range, pitch density and the pitch value of the lowest note; The beat data information is used to determine the rhythm type corresponding to the music accompaniment according to the number of music beats per unit time, and the rhythm type includes slow, medium and fast; The chord data information is used to assist the melody data information in determining the harmony of the notes through chord positions.

6. The method according to any one of claims 1 to 2, characterized in that: The method further comprises: performing mixing processing on the melody data information and the music accompaniment data to obtain a mixing result; Audio data rendering is performed on the mixing result to obtain a music result corresponding to the music data, wherein the music result uses the music accompaniment as accompaniment content.

7. The method according to claim 6, characterized in that The mixing process of the melody data information and the music accompaniment data to obtain a mixing result includes: Determining a music melody track corresponding to the melody data information and a music accompaniment track corresponding to the music accompaniment data; Under the condition that the audio tracks are aligned, the music melody track and the music accompaniment track are mixed to generate the mixing result.

8. A device for generating a music accompaniment, characterized in that: The device comprises: A data acquisition module is used to acquire music data, where the music data is divided into a plurality of music beats, and the music beats include music notes; An information extraction module is configured to determine the beat data information corresponding to the music data based on the number of the multiple music beats in a unit time, wherein the beat data information is used to characterize the changing speed of the multiple music beats; perform music information extraction on the music data to determine the chord track and melody track corresponding to the music data, wherein the chord track is used to describe the sound group constructed by the multiple music notes in the form of chords, and the melody track is used to describe the main melody constructed by the multiple music notes according to the time sequence relationship; respectively determine the number of bars in the melody track and the number of bars in the chord track, and obtain a first number corresponding to the melody track and the chord track. a second number corresponding to a track, the first number being used to represent the number of bars in the melody track, and the second number being used to represent the number of bars in the chord track; adjusting the second number of the chord track based on the first number to obtain an adjusted chord track as chord data information, the chord data information being used to describe chord units extracted based on the beat of the music; performing melody transformation processing on the melody track based on the note pitches corresponding to the multiple music notes in the melody track to obtain an updated melody track as melody data information, the melody data information being used to describe note changes between the multiple music notes; a data generation module, configured to use the melody data information as a generation restriction condition for the music accompaniment, and to generate music accompaniment data using the beat data information and the chord data information, wherein the melody data information is used to limit the accompaniment melody of the music accompaniment using the music melody of the music data, and the music accompaniment data describes the accompaniment of the music data using music parameters; The accompaniment generation module is used to render audio data based on the music accompaniment data to obtain the music accompaniment corresponding to the music data.

9. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one program, and the at least one program is loaded and executed by the processor to implement the method for generating a music accompaniment according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The storage medium stores at least one program, and the at least one program is loaded and executed by the processor to implement the method for generating a music accompaniment according to any one of claims 1 to 7.

11. A computer program product, characterized in that The method comprises a computer program or an instruction, which, when executed by a processor, implements the method for generating a music accompaniment as claimed in any one of claims 1 to 7.

Images