A method, apparatus, and computer-readable storage medium for generating music information

Abstract

The embodiments of the present invention disclose a method, device and computer-readable storage medium for generating music information. After obtaining a music information sample, the embodiments of the present invention parse the music information sample to obtain a set of music events corresponding to the music information sample, then use a preset music model to screen out a target music event corresponding to each audio track in the music event set, and construct an initial music sequence corresponding to the audio track based on the target music event. Then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information. This solution can improve the accuracy of music information generation for various types of music. The embodiments of the present invention can be applied to various scenarios such as cloud technology, artificial intelligence, and smart transportation.

Description

Technical Field

[0001] The present invention relates to the field of communication technology, and in particular to a music information generating method, device and computer-readable storage medium. Background Art

[0002] In recent years, with the rapid development of Internet technology, the application of neural networks to music generation has also made great progress, especially for music information of multiple tracks. Existing music information generation methods often require modeling and decoding for each track to generate multi-track music information.

[0003] During the research and practice of the prior art, the inventors of the present invention discovered that the existing music information generation method often creates multiple tracks when modeling the audio track. During the decoding process, multiple decoders are required for decoding, and the decoding processes are independent of each other. The coordination of the various audio tracks is often not good, which makes the generated music information sound like many discordant notes or harmonies, etc., thereby reducing the accuracy of the music information generation. Summary of the Invention

[0004] The embodiments of the present invention provide a music information generating method, device and computer-readable storage medium, which can improve the accuracy of generating music information.

[0005] A method for generating music information, comprising:

[0006] Acquire a music information sample, wherein the music information sample includes music information of multiple tracks;

[0007] Parsing the music information sample to obtain a set of music events corresponding to the music information sample;

[0008] Using a preset music model to filter out a target music event corresponding to each track in the music event set, and constructing an initial music sequence corresponding to the track based on the target music event;

[0009] fusing the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample;

[0010] The preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate target music information.

[0011] Accordingly, an embodiment of the present invention provides a music information generating device, comprising:

[0012] an acquiring unit, configured to acquire a music information sample, wherein the music information sample includes music information of a plurality of tracks;

[0013] a parsing unit, configured to parse the music information sample to obtain a set of music events corresponding to the music information sample;

[0014] a construction unit, configured to select a target music event corresponding to each music track from the music event set using a preset music model, and construct an initial music sequence corresponding to the music track based on the target music event;

[0015] a fusion unit, configured to fuse the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample;

[0016] A generating unit is used to converge the preset music model based on the music sequence to obtain a trained music model, and use the trained music model to generate target music information.

[0017] Optionally, in some embodiments, the construction unit can be specifically used to fuse the target music events according to the event type of the target music events to obtain a fused music event; and construct an initial music sequence corresponding to the audio track based on the fused music event.

[0018] Optionally, in some embodiments, the construction unit can be specifically used to classify the target music events to obtain track events, beat events and note events; fuse the beat events to obtain fused beat events, and fuse the note events to obtain fused note events; and use the track events, fused beat events and fused note events as fused music events.

[0019] Optionally, in some embodiments, the construction unit can be specifically used to obtain the first position information of the beat position event and the second position information of the music speed event respectively; match the beat position event and the music speed event according to the first position information and the second position information; and fuse the successfully matched beat position event and music speed event to obtain a fused beat event.

[0020] Optionally, in some embodiments, the construction unit can be specifically used to query the target music speed event with the same position as the beat position event in the music speed event based on the first position information and the second position information; when the target music speed event exists in the music speed event, it is determined that the target music speed event and the beat position event are successfully matched.

[0021] Optionally, in some embodiments, the construction unit can be specifically used to obtain the first note information of the pitch event, the second note information of the duration event, and the third note information of the velocity event respectively; match the pitch event, duration event, and velocity event according to the first note information, the second note information, and the third note information; and fuse the successfully matched pitch event, duration event, and velocity event to obtain a fused note event.

[0022] Optionally, in some embodiments, the construction unit can be specifically used to respectively obtain the third position information of the audio track event and the fourth position information of the fused note event; sort the audio track event, the fused beat event and the fused note event based on the position information, the third position information and the fourth position information of the fused beat event; and arrange the audio track event, the fused beat event and the fused note event according to the first sorting result to obtain the initial music sequence corresponding to the audio track.

[0023] Optionally, in some embodiments, the fusion unit can be specifically used to determine the starting position of the corresponding initial music sequence based on the third position information of the audio track event; filter out the bar position event in the music event set, and obtain the fifth position information of the bar position event; based on the starting position and the fifth position information, fuse the bar position event and the initial music sequence to obtain the music sequence corresponding to the music information sample.

[0024] Optionally, in some embodiments, the fusion unit can be specifically used to sort the measure position events and the initial music sequence according to the starting position and the fifth position information; based on the second sorting result, the measure position events and the initial music sequence are arranged to obtain a music sequence corresponding to the music information sample.

[0025] Optionally, in some embodiments, the generation unit can be specifically used to decode the music sequence using the preset music model to obtain predicted music information; determine the loss information of the music information sample based on the predicted music information and the music information sample; and converge the preset music model based on the loss information to obtain a trained music model.

[0026] Optionally, in some embodiments, the generation unit can be specifically used to receive a music information generation request, where the music information generation request carries basic music information; and based on the type of the basic music information, use the trained music model to generate target music information corresponding to the music information generation request.

[0027] Optionally, in some embodiments, the generation unit can be specifically used to, when the type of the basic music information is a music start identifier, use the trained music model to generate music information, and obtain the target music information corresponding to the music information generation request; when the type of the basic music information is prelude music information, use the trained music model to generate music information that matches the prelude music information, and obtain the target music information corresponding to the music information generation request.

[0028] In addition, an embodiment of the present invention further provides an electronic device, including a processor and a memory, wherein the memory stores an application program, and the processor is configured to run the application program in the memory to implement the music information generation method provided by the embodiment of the present invention.

[0029] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores multiple instructions, and the instructions are suitable for a processor to load to execute the steps in any music information method provided by the embodiment of the present invention.

[0030] After obtaining a music information sample, the embodiment of the present invention parses the music information sample to obtain a set of music events corresponding to the music information sample, then uses a preset music model to screen out target music events corresponding to each audio track in the music event set, and constructs an initial music sequence corresponding to the audio track based on the target music event, then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate target music information; since this scheme fuses the music information of multiple audio tracks into one music sequence when training the music model, only one decoder can be used to generate music information of multiple audio tracks, thereby improving the accuracy of music information generation. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0032] Figure 1 Schematic diagram of a scenario of a music information generating method provided by an embodiment of the present invention;

[0033] Figure 2 1 is a flow chart of a method for generating music information provided by an embodiment of the present invention;

[0034] Figure 3is a schematic diagram of an initial music sequence provided by an embodiment of the present invention;

[0035] Figure 4 is a schematic diagram of a music sequence of one measure provided by an embodiment of the present invention;

[0036] Figure 5 is a schematic diagram of the network structure of a decoding network provided by an embodiment of the present invention;

[0037] Figure 6 is a schematic diagram of training a preset music model and generating target music information provided by an embodiment of the present invention;

[0038] Figure 7 is another flowchart of the music information generating method provided by an embodiment of the present invention;

[0039] Figure 8 is a structural diagram of a music information generating device provided by an embodiment of the present invention;

[0040] Figure 9 It is a structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making any creative efforts shall fall within the scope of protection of the present invention.

[0042] The embodiment of the present invention provides a method, device and computer-readable storage medium for generating music information, wherein the music information generating device can be integrated into an electronic device, which can be a server, a terminal or other device.

[0043] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, tablet computer, laptop computer, desktop computer, smart speaker, smart watch, smart voice interaction device, smart home appliance, car terminal, etc., but is not limited to these. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and this application does not limit this.

[0044] For example, see Figure 1 Taking the integration of a music information generating device in an electronic device as an example, after obtaining a music information sample, the electronic device parses the music information sample to obtain a set of music events corresponding to the music information sample, and then uses a preset music model to screen out the target music events corresponding to each audio track in the music event set, and constructs an initial music sequence corresponding to the audio track based on the target music event, and then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, and the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information, thereby improving the accuracy of the generated music information.

[0045] Among them, music information can be understood as computer-recognizable information containing music, for example, it can be a description of the music to be played in the form of bytes. The music information can be a file collected through a Musical Instrument Digital Interface (MIDI), also known as a MIDI file, which can be played directly through an audio player, or the music file can be converted into a conventional audio format for playback. In addition, there is still a certain difference between music information and audio information. Audio information mainly contains audio signals, while music information mainly contains bytes describing music information. Audio information can be converted into music information.

[0046] It should be noted that the order of description of the following embodiments is not intended to limit the preferred order of the embodiments.

[0047] This embodiment will be described from the perspective of a music information generating device, which can be specifically integrated into an electronic device, which can be a server or a terminal; wherein the terminal can include a tablet computer, a laptop computer, a personal computer (PC), a wearable device, a virtual reality device, an intelligent voice interaction device, a smart home appliance, a car terminal or other smart devices that can generate music information.

[0048] A method for generating music information, comprising:

[0049] After obtaining the music information sample, which includes music information of multiple audio tracks, the music information sample is parsed to obtain a set of music events corresponding to the music information sample. Then, a preset music model is used to screen out target music events corresponding to each audio track in the music event set, and an initial music sequence corresponding to the audio track is constructed based on the target music event. According to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample. Then, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information.

[0050] like Figure 2 As shown, the specific process of the music information generation method is as follows:

[0051] 101. Obtain music information sample.

[0052] The music information sample includes music information of multiple tracks. The so-called music information can be understood as computer-recognizable information containing music, for example, it can be the music information (MIDI file) that is described in the form of bytes to be played.

[0053] There are many ways to obtain music information samples, which can be as follows:

[0054] For example, it is possible to directly receive music information samples uploaded or sent by users through the terminal, or it is possible to obtain music information and intercept music information of a preset time or preset memory size from the music information to obtain music information samples, or it is possible to receive audio information uploaded or sent by users through the terminal and convert the audio information into music information to obtain music information samples, or, when the memory of the music information samples is large or the number is large, it is possible to receive a music generation request sent by the terminal, which carries the storage address of the music information sample, and obtain the music information sample according to the storage address.

[0055] 102. Analyze the music information sample to obtain a set of music events corresponding to the music information sample.

[0056] The music event set may include multiple music events. The so-called music event may be an event corresponding to the music action that produces the music contained in the music information sample. For example, taking the music information sample as a MIDI file as an example, the music event may be a MIDI event, and the music event set may be a MIDI event set. The MIDI event set may include multiple music event elements, such as a track event.<TRACK_X> , Beat Position Event <beat>, Music Speed Event<TEMPO_X> , pitch events<PITCH_X> , duration events<DURATION_X> , force events<VELOCITY_X> and bar position events <bar>Track Events<TRACK_X> Indicates the starting position of track X, beat position event <beat>Indicates the starting position of the beat, music speed event<TEMPO_X> Indicates that the speed of the music here is X, the pitch event<PITCH_X> Indicates that the note's velocity is X, and the duration is X.<DURATION_X> Indicates that the duration of the note is X, the intensity event<VELOCITY_X> Indicates that the note's velocity is X, and the bar position event <bar>Indicates the starting position of a measure, etc.

[0057] There are many ways to parse the music information sample, which can be as follows:

[0058] For example, by identifying the track block data in the music information, the basic music event information corresponding to each track can be obtained. Based on the basic music event, the basic music event of each track is generated. By converting the basic music event into a music event, a music event set corresponding to the music information sample can be obtained.

[0059] Among them, taking the music information sample as a MIDI file as an example, the basic music event can be a basic MIDI event and a meta-event. There are many ways to convert the basic music event into a music event. For example, the basic music event can be converted into a music event through a conversion tool. There are many conversion tools, for example, pretty midi (a MIDI parsing tool) or other conversion tools, etc.

[0060] 103. Use a preset music model to filter out a target music event corresponding to each track in the music event set, and construct an initial music sequence corresponding to the track based on the target music event.

[0061] The initial music sequence may be a sequence obtained by arranging the music events in each track in a certain order, and each token in the sequence is a music event.

[0062] There are many ways to use the preset music model to filter out the target music event of each track from the music event set, which can be as follows:

[0063] For example, a track conversion network with a preset music model can be used to filter out music events other than measure position events in a music event set to obtain candidate music events. The candidate music events are then classified according to the track identifiers of the tracks, thereby obtaining the target music events for each track.

[0064] After filtering out the target music events of each audio track, the initial music sequence corresponding to the audio track can be constructed based on the target music events. There are many ways to construct the initial music sequence. For example, according to the target music event type, the target music events are fused to obtain fused music events, and the initial music sequence corresponding to the audio track is constructed based on the fused music events.

[0065] Among them, there are many ways to fuse the target music events. For example, the target music events can be classified to obtain track events, beat events and note events, the beat events can be fused to obtain fused beat events, and the note events can be fused to obtain fused note events, and the track events, fused beat events and fused note events can be used as fused music events.

[0066] Among them, the beat event can include the beat position event and the music speed event. There are many ways to fuse the beat events. For example, the first position information of the beat position event and the second position information of the music speed event are obtained respectively, and the beat position event and the music speed event are matched according to the first position information and the second position information. The successfully matched beat position event and music speed event are fused to obtain the fused beat event.

[0067] The first position information may indicate the starting position corresponding to the beat position event, and the second position information may indicate the position corresponding to the music speed event. Therefore, it can be understood that the music event corresponding to the music speed at the second position is the music speed event. There are multiple ways to match the beat position information and the music speed event based on the first position information and the second position information. For example, based on the first position information and the second position information, a target music speed event with the same position as the beat position event can be searched in the music speed event. When the target music speed event exists in the music speed time, it is determined that the target music speed event and the beat position event are successfully matched.

[0068] Among them, the successful matching here can be understood as the beat position event and music speed event at the same position. There are many ways to fuse the successfully matched beat position event and music speed event. For example, the successfully matched beat position event and music speed event can be directly spliced or added to obtain the fused beat event, or the weighted coefficients of the successfully matched beat position event and music speed event can be obtained respectively. Based on the weighted coefficients, the beat position event and music speed event are weighted respectively to obtain the weighted beat position event and the weighted music speed event. The weighted beat position event and the weighted music speed event are fused to obtain the fused beat event. The fused beat event can be one or more. When there are multiple beat positions, the number of fused beat events can be multiple. When there is one beat position, the number of fused beat events can be one.

[0069] Among them, note events can include pitch events, duration events and velocity events. There are many ways to fuse note events. For example, the first note information of the pitch event, the second note information of the duration event and the third note information of the velocity event can be obtained respectively. According to the first note information, the second note information and the third note information, the pitch event, duration event and velocity event are matched, and the successfully matched pitch events, duration events and velocity events are fused to obtain the fused note event.

[0070] Among them, the note information can be understood as the information of the note corresponding to the note event, that is, on which note the note event occurs. According to the first note information, the second note information and the third note information, matching the pitch event, the duration event and the intensity event can be understood as matching the pitch event, intensity event and duration event of the same note. The specific matching and fusion can refer to the matching and fusion method of the beat position event and the music speed event, which will not be repeated here, so as to obtain the fused note event. The number of the fused note events can be one or more. When there are multiple note information, the fused note events can be multiple. When there is only one note information, it means that there is one note in the audio track, and the number of the fused note events can be one.

[0071] After the target music events are fused, the initial music sequence corresponding to the audio track can be constructed based on the fused music events. There are many ways to construct it. For example, the third position information of the audio track event and the fourth position information of the fused note event can be obtained respectively. Based on the position information, third position information and fourth position information of the fused beat event, the audio track event and the fused beat event are sorted respectively to obtain a first sorting result. According to the first sorting result, the audio track event, the fused beat event and the fused note event are arranged in memory to obtain the initial music sequence corresponding to the audio track.

[0072] Among them, according to the sorting results, there are many ways to arrange the track events, the fused beat events and the fused note events. For example, the event number of the track events, the fused beat events and the fused note events can be obtained, and based on the event number, a basic blank sequence is constructed according to the preset sequence structure. The track events, the fused beat events and the fused note events are added as tokens to the basic blank sequence according to the sorting results to obtain the initial music sequence corresponding to the track. Specifically, Figure 3 shown.

[0073] 104. According to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample.

[0074] The music sequence may be a complete sequence corresponding to the music information sample, and the music sequence may include initial music sequences corresponding to multiple tracks.

[0075] There are multiple ways to fuse the initial music sequences according to their starting positions, which can be as follows:

[0076] For example, the starting position of the corresponding initial music sequence can be determined based on the third position information of the audio track event, the bar position event can be filtered in the music event set, and the fifth position information of the bar position event can be obtained. Based on the starting position and the fifth position information, the bar position event and the initial music sequence can be fused to obtain the music sequence corresponding to the music information sample.

[0077] Among them, there are multiple ways to determine the starting position of the corresponding initial music sequence based on the third position information of the audio track event. For example, the position corresponding to the third position information can be directly used as the starting position of the initial music sequence of the corresponding audio track, or the third position information of the audio track event can be converted to obtain the starting position of the initial music sequence.

[0078] After determining the starting position of the initial music sequence, the bar position event and the initial music sequence can be fused according to the starting position and the fifth position event of the bar position event. There are many ways to fuse them. For example, according to the starting position and the fifth position information, the initial music sequence of the bar position event is sorted to obtain a second sorting result. Based on the second sorting result, the bar position event and the initial music sequence are arranged to obtain a music sequence corresponding to the music information sample.

[0079] Among them, there are many ways to arrange the bar position event and the initial music sequence based on the second sorting result. For example, the initial music sequence can be arranged based on the second sorting result to obtain a basic music sequence, and the addition position of the bar position event in the basic music sequence is determined according to the second sorting result. According to the addition position, the bar position event is added to the basic music sequence to obtain a music sequence corresponding to the music information sample. A complete piece of music may include multiple bars. Taking the music sequence of one bar as an example, Figure 4 As shown, the bar position event can be added between the initial music sequence and the initial music sequence, and two bar position events can form a bar in a piece of music. Figure 4 As you can see, this measure can include four beats and two tracks. The information of each track is arranged in order, and the representation method of each track is the same, all based on the following pattern: <track> <beat>…Notes… <beat>...Notes.... First is a<TRACK_X> token, indicating that the following tokens all belong to track X, followed by a <beat>, refers to the starting position of a beat, followed by a series of notes (indicated by green boxes in the figure), each note has three pieces of information (pitch, duration, and intensity). <beat>If there are multiple notes after it, it means that these notes should be played at the same time.

[0080] 105. Converging the preset music model based on the music sequence to obtain a trained music model, and using the trained music model to generate target music information.

[0081] There are many ways to converge the preset music model, which can be as follows:

[0082] For example, a preset music model can be used to decode the music sequence to obtain predicted music information. Based on the predicted music information and the music information sample, the loss information of the music information sample can be determined. Based on the loss information, the preset music model can be converged to obtain a trained music model.

[0083] There are many ways to decode the music sequence. For example, a decoding network with a preset music model can be used to decode the music sequence to obtain predicted music information. Since the music sequence contains multiple tracks, the length of the music sequence is usually thousands or even tens of thousands. Therefore, it is necessary to select an attention network that models longer sequences. Linear Transformer (attention network for long sequence modeling) or other attention networks can be used. In addition, since the obtained music sequence is not one-dimensional, for example, a note event requires three-dimensional information to represent. Therefore, different headers are used to encode and decode information of each dimension. The network structure of the decoding network is as follows: Figure 5 shown.

[0084] Among them, there are many ways to converge the preset music model based on the loss information. For example, based on the loss information, the gradient descent algorithm can be used to update the network parameters of the preset music model to obtain the trained music model, or other algorithms can be used to update the network parameters of the preset music model based on the loss information to obtain the trained music model.

[0085] Among them, when training the music model, it can be regarded as inputting the music information sample of the music music (music on the score) containing multiple instruments (multiple tracks) into the encoding network (Multi-track event-based Representation, MTEBR) of the preset music model, and using MTEBR to convert the music information sample into the music sequence corresponding to the music information sample. Each token of the sequence is a MIDI event. This conversion process can be regarded as encoding the music information sample, and then using the decoding network (LinearTransformer) of the preset music model to decode it to obtain the predicted music information, and then determine the loss information based on the predicted music information and the music information sample, and converge the preset music model based on the loss information to obtain the trained music model, and then use the trained music model to generate the target music information, such as Figure 6 shown.

[0086] After obtaining the trained music model, the target music information can be generated based on the trained music model. There are many ways to generate the target music information, for example, receiving a music information generation request, which carries basic music information, and using the trained music model to generate the target music information corresponding to the music information generation request according to the type of the basic music information.

[0087] Among them, according to the type of basic music information, there can be multiple ways to use the trained music model to generate the target music information corresponding to the music information generation request. For example, when the basic music information is a music start identifier, the trained music model is used to generate music information to obtain the target music information corresponding to the music information generation request. When the basic music information is prelude music information, the trained music model is used to generate music information that matches the prelude music information to obtain the target music information corresponding to the music information generation request.

[0088] The types of basic music information may include a music start identifier and prelude music information. The so-called music start identifier can be identification information that triggers the trained music model to generate target music information. The prelude music information can be music information corresponding to the prelude music, which is usually the prelude part of the music. In this case, the generated target music information can be music information that matches the prelude music. Depending on the type of basic music information, the generated target music information can be the same or different.

[0089] From the above, it can be seen that after obtaining the music information sample, the embodiment of the present application parses the music information sample to obtain a set of music events corresponding to the music information sample, and then uses a preset music model to screen out the target music events corresponding to each audio track in the music event set, and constructs an initial music sequence corresponding to the audio track based on the target music event, and then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information; since this scheme fuses the music information of multiple audio tracks into one music sequence when training the music model, only one decoder can be used to generate music information of multiple audio tracks, thereby improving the accuracy of music information generation.

[0090] The method described in the above embodiment will be further described in detail below with examples.

[0091] In this embodiment, the music information generating device is specifically integrated into an electronic device, the electronic device may be a server, the music information sample is a MIDI file sample, and the music event is a MIDI event.

[0092] like Figure 7 As shown, a method for generating music information, the specific process is as follows:

[0093] 201. The server obtains a MIDI file sample.

[0094] For example, the server can directly receive MIDI file samples uploaded or sent by the user through the terminal, or it can obtain the original MIDI file, cut out the MIDI file of a preset time or preset memory size from the original MIDI file, and obtain the MIDI file sample, or it can receive audio information uploaded or sent by the user through the terminal, convert the audio information into a MIDI file, and thus obtain the MIDI file sample, or, when the MIDI file sample memory is large or the number is large, it can also receive a music generation request sent by the terminal, which carries the storage address of the MIDI file sample, and obtain the MIDI file sample according to the storage address.

[0095] 202. The server parses the MIDI file sample to obtain a MIDI event set corresponding to the MIDI file sample.

[0096] For example, the server can identify the track block data in the MIDI file through pretty midi, obtain the basic MIDI event information corresponding to each track, generate the basic MIDI event of each track based on the basic MIDI event, and convert the basic MIDI event into MIDI events.

[0097] 203. The server uses a preset music model to filter out target MIDI events corresponding to each track from the basic MIDI event set.

[0098] For example, the server can use a track conversion network with a preset music model to filter out MIDI events other than measure position events in the MIDI event set, obtain candidate MIDI events, and classify the candidate MIDI events according to the track identifier of the track, so as to obtain the target MIDI event of each track.

[0099] 204. The server constructs an initial music sequence corresponding to the audio track based on the target MIDI event.

[0100] For example, the server classifies the target MIDI events into track events, beat events, and note events. The server obtains first position information of the beat position event and second position information of the music tempo event, respectively. Based on the first position information and the second position information, the server searches the music tempo events for a target music tempo event with the same position as the beat position event. If the target music tempo event exists in the music tempo time, it is determined that the target music tempo event and the beat position event are successfully matched.

[0101] The server can directly splice or add the successfully matched beat position events and music speed events to obtain a fused beat event, or it can obtain the weighted coefficients of the successfully matched beat position events and music speed events respectively, and weight the beat position events and music speed events based on the weighted coefficients to obtain weighted beat position events and weighted music speed events. The weighted beat position events and weighted music speed events are fused to obtain a fused beat event. The fused beat event can be one or more. When there are multiple beat positions, the number of fused beat events can be multiple. When there is one beat position, the number of fused beat events can be one.

[0102] The server obtains the first note information of the pitch event, the second note information of the duration event, and the third note information of the velocity event respectively, matches the pitch event, duration event, and velocity event according to the first note information, the second note information, and the third note information, and merges the successfully matched pitch event, duration event, and velocity event to obtain a merged note event.

[0103] The server obtains the third position information of the audio track event and the fourth position information of the fused note event respectively, and sorts the audio track event and the fused beat event respectively based on the position information, the third position information and the fourth position information of the fused beat event.

[0104] The server obtains the number of track events, fused beat events, and fused note events, and based on the number of events, constructs a basic blank sequence according to a preset sequence structure, adds the track events, fused beat events, and fused note events as tokens to the basic blank sequence according to the first sorting result, and obtains the initial music sequence corresponding to the track

[0105] 205. The server merges the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the MIDI file sample.

[0106] For example, the server may directly use the position corresponding to the third position information as the starting position of the initial music sequence of the corresponding audio track, or convert the third position information of the audio track event to obtain the starting position of the initial music sequence.

[0107] The server filters the bar position event from the MIDI event collection and obtains the fifth position information of the bar position event. Based on the starting position and the fifth position information, the server sorts the initial music sequence of the bar position event to obtain a second sorting result. Based on the second sorting result, the server determines the position at which the bar position event should be added to the basic music sequence. Based on the addition position, the server adds the bar position event to the basic music sequence to obtain a music sequence corresponding to the MIDI file sample.

[0108] 206. The server converges the preset music model based on the music sequence to obtain a trained music model.

[0109] For example, the server can use a linear Transformer to decode the music sequence to obtain a predicted MIDI file. Based on the predicted MIDI file and the MIDI file sample, the server can determine the loss information of the MIDI file sample. Based on the loss information, the server can use a gradient descent algorithm to update the network parameters of the preset music model to obtain a trained music model. Alternatively, other algorithms can be used to update the network parameters of the preset music model based on the loss information to obtain a trained music model.

[0110] 207. The server generates a target MIDI file using the trained music model.

[0111] For example, the server receives a MIDI file generation request, which carries a basic MIDI file. When the basic MIDI file is a music start identifier, the trained music model is used to generate the MIDI file, and the target MIDI file corresponding to the MIDI file generation request is obtained. When the basic MIDI file is a prelude MIDI file, the trained music model is used to generate a MIDI file that matches the prelude MIDI file, and the target MIDI file corresponding to the MIDI file generation request is obtained.

[0112] From the above, it can be seen that after obtaining the MIDI file sample, the server of this embodiment parses the NIDI file sample to obtain the MIDI event set corresponding to the MIDI file sample, and then uses the preset music model to filter out the target MIDI events corresponding to each audio track in the basic MIDI event set, and constructs the initial music sequence corresponding to the audio track based on the target MIDI events. Then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain the music sequence corresponding to the music information sample, the preset music model is converged based on the music sequence to obtain the trained music model, and the trained music model is used to generate the target MIDI file; since this scheme fuses the music information of multiple audio tracks into one music sequence when training the music model, only one decoder can be used to generate the music information of multiple audio tracks, thereby improving the accuracy of music information generation.

[0113] In order to better implement the above method, an embodiment of the present invention also provides a music information generating device, which can be integrated into an electronic device, such as a server or terminal, and the terminal may include a tablet computer, a laptop computer and / or a personal computer, an intelligent voice interaction device, a smart home appliance, a vehicle-mounted terminal, etc.

[0114] For example, Figure 8 As shown, the music information generating device may include an acquiring unit 301, a parsing unit 302, a constructing unit 303, a fusing unit 304, and a generating unit 305, as follows:

[0115] (1) Acquisition unit 301;

[0116] The acquisition unit 301 is used to acquire a music information sample, where the music information sample includes music information of multiple tracks.

[0117] For example, the acquisition unit 301 can be specifically used to directly receive music information samples uploaded or sent by the user through the terminal, or it can acquire music information, intercept music information of a preset time or preset memory size from the music information, and obtain a music information sample, or it can receive audio information uploaded or sent by the user through the terminal, convert the audio information into music information, and thus obtain a music information sample, or, when the music information sample memory is large or the number is large, it can also receive a music generation request sent by the terminal, which carries the storage address of the music information sample, and obtain the music information sample according to the storage address.

[0118] (2) parsing unit 302;

[0119] The parsing unit 302 is configured to parse the music information sample to obtain a set of music events corresponding to the music information sample.

[0120] For example, the parsing unit 302 can be specifically used to identify track block data in the music information, so as to obtain the basic music event information corresponding to each track, generate the basic music event of each track based on the basic music event, and convert the basic music event into a music event, so as to obtain the music event set corresponding to the music information sample.

[0121] (3) building unit 303;

[0122] The construction unit 303 is configured to select a target music event corresponding to each track from the music event set using a preset music model, and to construct an initial music sequence corresponding to the track based on the target music event.

[0123] For example, the construction unit 303 can be specifically configured to use a track conversion network of a preset music model to filter out music events other than bar position events from a music event set to obtain candidate music events, and classify the candidate music events according to the track identifier of the track, thereby obtaining a target music event for each track. The third position information of the track event and the fourth position information of the fused note event are respectively obtained, and the track events and the fused beat events are sorted based on the position information, the third position information, and the fourth position information of the fused beat event to obtain a first sorting result. Based on the first sorting result, the track events, the fused beat events, and the fused note events are memorized and arranged to obtain an initial music sequence corresponding to the track.

[0124] (4) fusion unit 304;

[0125] The fusion unit 304 is configured to fuse the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample.

[0126] For example, the fusion unit 304 can be specifically used to determine the starting position of the corresponding initial music sequence based on the third position information of the audio track event, filter the bar position event in the music event set, and obtain the fifth position information of the bar position event. According to the starting position and the fifth position information, the bar position event and the initial music sequence are fused to obtain the music sequence corresponding to the music information sample.

[0127] (5) generating unit 305;

[0128] The generating unit 305 is configured to converge the preset music model based on the music sequence to obtain a trained music model, and use the trained music model to generate target music information.

[0129] For example, the generation unit 306 may be configured to decode the music sequence using a preset music model to obtain predicted music information, determine loss information of the music information sample based on the predicted music information and the music information sample, and converge the preset music model based on the loss information to obtain a trained music model. The generated music information generation request may include basic music information, and the trained music model may be used to generate target music information corresponding to the generated music information request based on the type of the basic music information.

[0130] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can be found in the previous method embodiments and will not be repeated here.

[0131] From the above, it can be seen that after obtaining the music information sample, this embodiment parses the music information sample to obtain a set of music events corresponding to the music information sample, and then uses a preset music model to screen out the target music events corresponding to each audio track in the music event set, and constructs an initial music sequence corresponding to the audio track based on the target music event, and then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, and the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information; since this scheme fuses the music information of multiple audio tracks into one music sequence when training the music model, only one decoder can be used to generate music information of multiple audio tracks, thereby improving the accuracy of music information generation.

[0132] An embodiment of the present invention further provides an electronic device, such as Figure 9 , which shows a schematic structural diagram of an electronic device involved in an embodiment of the present invention, specifically:

[0133] The electronic device may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will understand that Figure 9 The electronic device structure shown in the figure does not constitute a limitation of the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange components differently.

[0134] Processor 401 is the control center of the electronic device, connecting the various parts of the entire electronic device using various interfaces and lines. By running or executing software programs and / or modules stored in memory 402 and accessing data stored in memory 402, it performs various functions of the electronic device and processes data, thereby performing overall testing of the electronic device. Optionally, processor 401 may include one or more processing cores; preferably, processor 401 may integrate an application processor and a modem processor, wherein the application processor primarily processes the operating system, user interface, and application programs, and the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into processor 401.

[0135] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0136] The electronic device also includes a power supply 403 for supplying power to various components. Preferably, the power supply 403 can be logically connected to the processor 401 via a power management system, thereby enabling the power management system to manage charging, discharging, and power consumption. The power supply 403 can also include one or more DC or AC power supplies, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and other arbitrary components.

[0137] The electronic device may further include an input unit 404, which may be configured to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0138] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the electronic device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402 to implement various functions as follows:

[0139] After obtaining the music information sample, which includes music information of multiple audio tracks, the music information sample is parsed to obtain a set of music events corresponding to the music information sample. Then, a preset music model is used to screen out target music events corresponding to each audio track in the music event set, and an initial music sequence corresponding to the audio track is constructed based on the target music event. According to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample. Then, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information.

[0140] For example, the electronic device can directly receive music information samples uploaded or sent by a user through a terminal, or can obtain music information and extract music information of a preset time or preset memory size from the music information to obtain the music information sample. Alternatively, the electronic device can receive audio information uploaded or sent by a user through a terminal and convert the audio information into music information to obtain the music information sample. Alternatively, when the memory of the music information sample is large or the number of music information samples is large, the electronic device can also receive a music generation request sent by the terminal, which carries the storage address of the music information sample and obtains the music information sample based on the storage address. By identifying the track block data in the music information, the basic music event information corresponding to each track can be obtained. Based on the basic music event, the basic music event of each track is generated. The basic music event is converted into a music event to obtain a music event set corresponding to the music information sample. The track conversion network using a preset music model filters out music events other than the bar position event from the music event set to obtain candidate music events. The candidate music events are classified according to the track identifier of the track to obtain the target music event for each track. The third position information of the track event and the fourth position information of the fused note event are obtained respectively. Based on the position information, third position information, and fourth position information of the fused beat event, the track events and the fused beat event are sorted respectively to obtain a first sorting result. Based on the first sorting result, the track events, the fused beat events, and the fused note events are memorized and arranged to obtain an initial music sequence corresponding to the track. Based on the third position information of the track event, the starting position of the corresponding initial music sequence is determined. The measure position event is selected from the music event set, and the fifth position information of the measure position event is obtained. Based on the starting position and the fifth position information, the measure position event and the initial music sequence are fused to obtain a music sequence corresponding to the music information sample. A preset music model is used to decode the music sequence to obtain predicted music information. Based on the predicted music information and the music information sample, the loss information of the music information sample is determined. Based on the loss information, the preset music model is converged to obtain a trained music model. A music information generation request is received, the music information generation request carrying basic music information, and target music information corresponding to the music information generation request is generated using a trained music model according to the type of the basic music information.

[0141] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.

[0142] From the above, it can be seen that after obtaining the music information sample, the embodiment of the present invention parses the music information sample to obtain a set of music events corresponding to the music information sample, and then uses a preset music model to screen out the target music events corresponding to each audio track in the music event set, and constructs an initial music sequence corresponding to the audio track based on the target music event, and then, according to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information; since this scheme fuses the music information of multiple audio tracks into one music sequence when training the music model, only one decoder can be used to generate music information of multiple audio tracks, thereby improving the accuracy of music information generation.

[0143] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be accomplished by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0144] To this end, an embodiment of the present invention provides a computer-readable storage medium storing a plurality of instructions that can be loaded by a processor to execute the steps of any of the methods for generating music information provided in the embodiments of the present invention. For example, the instructions can execute the following steps:

[0145] After obtaining the music information sample, which includes music information of multiple audio tracks, the music information sample is parsed to obtain a set of music events corresponding to the music information sample. Then, a preset music model is used to screen out target music events corresponding to each audio track in the music event set, and an initial music sequence corresponding to the audio track is constructed based on the target music event. According to the starting position of the initial music sequence, the initial music sequence is fused to obtain a music sequence corresponding to the music information sample. Then, the preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate the target music information.

[0146] For example, the electronic device can directly receive music information samples uploaded or sent by a user through a terminal, or can obtain music information and extract music information of a preset time or preset memory size from the music information to obtain the music information sample. Alternatively, the electronic device can receive audio information uploaded or sent by a user through a terminal and convert the audio information into music information to obtain the music information sample. Alternatively, when the memory of the music information sample is large or the number of music information samples is large, the electronic device can also receive a music generation request sent by the terminal, which carries the storage address of the music information sample and obtains the music information sample based on the storage address. By identifying the track block data in the music information, the basic music event information corresponding to each track can be obtained. Based on the basic music event, the basic music event of each track is generated. The basic music event is converted into a music event to obtain a music event set corresponding to the music information sample. The track conversion network using a preset music model filters out music events other than the bar position event from the music event set to obtain candidate music events. The candidate music events are classified according to the track identifier of the track to obtain the target music event for each track. The third position information of the track event and the fourth position information of the fused note event are obtained respectively. Based on the position information, third position information, and fourth position information of the fused beat event, the track events and the fused beat event are sorted respectively to obtain a first sorting result. Based on the first sorting result, the track events, the fused beat events, and the fused note events are memorized and arranged to obtain an initial music sequence corresponding to the track. Based on the third position information of the track event, the starting position of the corresponding initial music sequence is determined. The measure position event is selected from the music event set, and the fifth position information of the measure position event is obtained. Based on the starting position and the fifth position information, the measure position event and the initial music sequence are fused to obtain a music sequence corresponding to the music information sample. A preset music model is used to decode the music sequence to obtain predicted music information. Based on the predicted music information and the music information sample, the loss information of the music information sample is determined. Based on the loss information, the preset music model is converged to obtain a trained music model. A music information generation request is received, the music information generation request carrying basic music information, and target music information corresponding to the music information generation request is generated using a trained music model according to the type of the basic music information.

[0147] The specific implementation of the above operations can be found in the previous embodiments and will not be repeated here.

[0148] The computer-readable storage medium may include a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0149] Since the instructions stored in the computer-readable storage medium can execute the steps in any music information generation method provided in the embodiments of the present invention, the beneficial effects that can be achieved by any music information generation method provided in the embodiments of the present invention can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0150] According to one aspect of the present application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in various optional implementations of the aforementioned aspects of generating music information or generating MIDI files.

[0151] The above is a detailed introduction to a music information generation method, device and computer-readable storage medium provided by an embodiment of the present invention. Specific examples are used herein to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for technical personnel in this field, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.< / beat> < / beat> < / beat> < / beat> < / bar> < / beat> < / bar> < / beat>

Claims

1. A method for generating music information, characterized in that: include: Acquire a music information sample, wherein the music information sample includes music information of multiple tracks; Parsing the music information sample to obtain a set of music events corresponding to the music information sample; A preset music model is used to filter out target music events corresponding to each track from the music event set, and an initial music sequence corresponding to the track is constructed based on the target music events, including: classifying the target music events to obtain track events, beat events, and note events; fusing the beat events to obtain fused beat events, and fusing the note events to obtain fused note events; using the track events, fused beat events, and fused note events as fused music events; and constructing the initial music sequence corresponding to the track based on the fused music events. fusing the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample; The preset music model is converged based on the music sequence to obtain a trained music model, and the trained music model is used to generate target music information.

2. The music information generating method according to claim 1, wherein: The beat events include beat position events and music speed events. The fusion of the beat events to obtain fused beat events includes: Respectively acquiring first position information of the beat position event and second position information of the music speed event; Matching the beat position event and the music speed event according to the first position information and the second position information; The successfully matched beat position events and music speed events are fused to obtain the fused beat events.

3. The music information generating method according to claim 2, wherein: Matching the beat position event and the music speed event according to the first position information and the second position information includes: searching, in the music tempo event, for a target music tempo event having the same position as the beat position event according to the first position information and the second position information; When the target music speed event exists in the music speed events, it is determined that the target music speed event is successfully matched with the beat position event.

4. The music information generating method according to claim 1, wherein The note event includes a pitch event, a duration event, and a velocity event. The step of fusing the note events to obtain a fused note event includes: Obtain the first note information of the pitch event, the second note information of the duration event, and the third note information of the velocity event respectively; matching the pitch event, the duration event, and the velocity event according to the first note information, the second note information, and the third note information; The successfully matched pitch events, duration events, and velocity events are fused to obtain a fused note event.

5. The music information generating method according to claim 1, wherein The constructing the initial music sequence corresponding to the audio track based on the fused music event includes: respectively obtaining the third position information of the track event and the fourth position information of the fused note event; sorting the track events, the fused beat events, and the fused note events based on the position information, the third position information, and the fourth position information of the fused beat events; According to the first sorting result, the audio track events, the fused beat events and the fused note events are arranged to obtain an initial music sequence corresponding to the audio track.

6. The music information generating method according to claim 5, wherein: The step of fusing the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample includes: Determining the starting position of the corresponding initial music sequence according to the third position information of the audio track event; Filtering out a bar position event from the music event set, and obtaining fifth position information of the bar position event; According to the starting position and the fifth position information, the measure position event and the initial music sequence are fused to obtain a music sequence corresponding to the music information sample.

7. The music information generating method according to claim 6, wherein: The step of fusing the measure position event and the initial music sequence according to the starting position and the fifth position information to obtain a music sequence corresponding to the music information sample includes: sorting the measure position events and the initial music sequence according to the starting position and the fifth position information; Based on the second sorting result, the measure position events and the initial music sequence are arranged to obtain a music sequence corresponding to the music information sample.

8. The music information generating method according to any one of claims 1 to 7, characterized in that: The step of converging the preset music model based on the music sequence to obtain a trained music model includes: Decoding the music sequence using the preset music model to obtain predicted music information; determining loss information of the music information sample based on the predicted music information and the music information sample; Based on the loss information, the preset music model is converged to obtain a trained music model.

9. The music information generating method according to any one of claims 1 to 7, characterized in that: The step of generating target music information by using the trained music model includes: receiving a music information generation request, wherein the music information generation request carries basic music information; According to the type of the basic music information, the trained music model is used to generate target music information corresponding to the music information generation request.

10. The music information generating method according to claim 9, wherein: Generating target music information corresponding to the music information generation request according to the type of the basic music information includes: When the type of the basic music information is a music start identifier, generating music information using the trained music model to obtain target music information corresponding to the music information generation request; When the type of the basic music information is prelude music information, the trained music model is used to generate music information that matches the prelude music information, and the target music information corresponding to the music information generation request is obtained.

11. A music information generating device, characterized in that: include: an acquiring unit, configured to acquire a music information sample, wherein the music information sample includes music information of a plurality of tracks; a parsing unit, configured to parse the music information sample to obtain a set of music events corresponding to the music information sample; A construction unit is configured to use a preset music model to filter out target music events corresponding to each track in the music event set, and construct an initial music sequence corresponding to the track based on the target music events, including: classifying the target music events to obtain track events, beat events, and note events; fusing the beat events to obtain fused beat events, and fusing the note events to obtain fused note events; using the track events, fused beat events, and fused note events as fused music events; and constructing the initial music sequence corresponding to the track based on the fused music events. a fusion unit, configured to fuse the initial music sequence according to the starting position of the initial music sequence to obtain a music sequence corresponding to the music information sample; A generating unit is used to converge the preset music model based on the music sequence to obtain a trained music model, and use the trained music model to generate target music information.

12. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to execute the steps in the music information generating method according to any one of claims 1 to 10.

13. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

14. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a plurality of instructions, which are suitable for being loaded by a processor to execute the steps in the music information generating method according to any one of claims 1 to 10.

Images