A music composition assisting apparatus

Abstract

The application discloses a music creation auxiliary device, comprising: a feature extraction module for extracting high-level features of an input melody, the high-level features comprising weight notes, weight chords, terminal chords and structure chords; a feature encoding and splicing module for encoding current notes, weight notes and weight chords, splicing the encoding vectors to obtain an encoding guide vector, and for encoding chords, terminal chords and structure chords of the previous beat, splicing the encoding vectors to obtain a decoding guide vector; and a chord generation module for taking the encoding guide vector as the input of an encoder of a Transformer model, taking the decoding guide vector as the input of a decoder of the Transformer model, and conducting forward conduction operation through the Transformer model to generate chords. The device can improve music quality and accuracy.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of music production, and particularly relates to a music creation auxiliary device. BACKGROUND

[0002] Artificial intelligence assisted composition refers to using artificial intelligence technology to predict the melody to a certain extent or help arrange chords compatible with the melody to assist composers in completing the creation task. Compared with artificial intelligence composition, artificial intelligence assisted composition has lower participation in creation, but it can well help composers reduce hesitation and stereotyped state in the creation process. Therefore, artificial intelligence assisted composition has great potential value and significance.

[0003] Existing auxiliary composition models can be roughly divided into data and rule-based composition and composition applying natural language processing technology. For example, Chinese patent application 201410582178.1 discloses a chord accompaniment generation method based on genetic algorithm, which generates single tone sequence chords by using genetic algorithm. This over-reliance on data and rules results in unsatisfactory generated chords.

[0004] With the rapid development of natural language processing and deep learning technology, researchers have also begun to try to apply natural language processing and deep learning technology to music chord generation. For example: PopMAG: Pop Music Accompaniment Generation written by Microsoft Asia Research Institute and Zhejiang University, on the basis of traditional music coding method MIDI, proposes an extended music coding method MuMIDI, which explicitly models the dependency between notes, thereby generating multi-track music in the form of a single sequence, solving the harmony problem between different tracks, and making the generated music more harmonious. However, PopMAG uses the Transformer model which has good performance in the field of natural language processing, and pays attention to the dependency between notes, but the researchers do not consider using professional music knowledge to analyze music, resulting in the model ignoring the high-level music features which are crucial to music, and not noticing the differences between natural language and music.

[0005] According to the controllable text generation Transformer model and the music generation under the condition constraint have the same effect, so some researchers begin to try to apply the Transformer model with condition guidance to the music generation field to improve the quality and controllability of the generated music. For example: MuseNet groups the information such as pitch, volume and instrument into a token and adds it to the training data. In the training process, the Transformer model can learn related information such as composition style, so that it can create on this basis. That is, MuseNet uses harmonic, melody and style and other shallow features to guide the generation of the Transformer model. Although MuseNet considers the characteristics of music to some extent, it still stays in the shallow features of music such as style, or the underlying data such as melody and harmony, and does not consider the high-level features of music, lacking the quantitative analysis of music based on music analysis theory to extract features. SUMMARY

[0006] In view of the above, the purpose of the present application is to provide a music creation auxiliary device, which extracts high-level features from a melody, and constructs a guide condition for the encoding and decoding stages of a Transformer model based on the high-level features, and uses the guide condition to realize deeper music chord guidance generation, and further improve the quality and accuracy of music.

[0007] To achieve the above-mentioned purpose of the application, an embodiment provides a music creation auxiliary device, comprising:

[0008] A feature extraction module is configured to extract high-level features of an input melody, wherein the high-level features include weighted notes, weighted chords, terminal chords and structural chords.

[0009] A feature encoding and splicing module is configured to encode the current notes, weighted notes and weighted chords, splice the encoding vectors to obtain an encoding guide vector, and encode the chords, terminal chords and structural chords of the previous beat and splice the encoding vectors to obtain a decoding guide vector.

[0010] A chord generation module is configured to use the encoding guide vector as the input of the encoder of the Transformer model, use the decoding guide vector as the input of the decoder of the Transformer model, and perform forward conduction operation on the Transformer model to generate chords.

[0011] In one embodiment, the feature extraction module comprises:

[0012] A weighted note extraction unit is configured to extract rhythm accents, cut notes and long notes in a melody, and select notes that are rhythm accents but not cut notes, or notes that are not rhythm accents but are cut notes and long notes as weighted notes.

[0013] a weight chord extraction unit configured to extract a chord composed of representative notes in the current melody segment starting from the weight note as the weight chord;

[0014] a terminal chord extraction unit configured to identify a terminal or semi-terminal type in the chord progression to determine a terminal chord;

[0015] a structure chord extraction unit configured to extract a non-inversion chord in the melody, and the current non-inversion chord is a structure chord if the non-inversion chord is a I, II, IV or V degree chord of the current mode tonality.

[0016] In one embodiment, the feature extraction module, the weight chord extraction unit, the extraction of the weight chord includes:

[0017] (a) constructing a chord reference table containing a plurality of chords;

[0018] (b) performing frequency statistics of the weighted pitch of all notes in the current melody segment starting from the weight note in a manner that the weight note is weighted as α and other notes are weighted as β to obtain a first weighted statistics result of the current melody segment, wherein α>β;

[0019] (c) concatenating the current melody segment starting from the weight note with all melody segments in the adjacent history group to form a melody sequence, and performing frequency statistics of the weighted pitch of all notes in the melody sequence in a manner that the weight note is weighted as α and other notes are weighted as β to obtain a second weighted statistics result of the melody sequence, wherein α>β;

[0020] (d) comparing the first weighted statistics result and the second weighted statistics result with the chord reference table respectively, and selecting a chord with the minimum editing cost from the chord reference table as the weight chord corresponding to the weight note, wherein the editing cost refers to the operation complexity of converting the weighted statistics result into a chord in the chord reference table, and the chord with the minimum editing cost contains notes representative of the current melody segment.

[0021] In one embodiment, selecting a chord with the minimum editing cost from the chord reference table as the weight chord corresponding to the weight note includes:

[0022] comparing the first weighted statistics result with the chord reference table, sorting the chords in the chord reference table in ascending order according to the editing cost to obtain a first chord sorting list, selecting chords with root notes of natural scales of the current mode tonality as candidate chords from the first chord sorting list, and selecting a candidate chord with the minimum editing cost as the first candidate weight chord;

[0023] The second weighted statistical result is compared with a chord comparison table, chords in the chord comparison table are sorted in ascending order according to an edit cost to obtain a second chord sorting list, chords with a root tone being a natural scale under a current mode tonality are selected as candidate chords from the second chord sorting list, and a candidate chord with a minimum edit cost is selected as a second candidate weighted chord from the candidate chords;

[0024] The minimum edit cost is selected from the first candidate weighted chord and the second candidate weighted chord to obtain a minimum weighted chord;

[0025] When the minimum weighted chord is selected from the first candidate weighted chord, the current melody is segmented into a new group, and the new group is taken as a neighboring historical group for weighted chord calculation at a next time; when the minimum weighted chord is selected from the second candidate weighted chord, the current melody segment is divided into the neighboring historical group to update the neighboring historical group.

[0026] In an embodiment, the feature encoding and concatenation module comprises:

[0027] The first feature encoding and concatenation unit is configured to encode the current note, the weighted tone, and the weighted chord, and concatenate the encoded vectors to obtain an encoded guide vector, and comprises:

[0028] The first feature encoding and concatenation unit is configured to encode the current note, the weighted tone, and the weighted chord, and concatenate the encoded vectors to obtain an encoded guide vector, and comprises:

[0029] First, the current note is embedded and encoded to obtain a note vector, and the weighted chord is embedded and encoded and then linearized to obtain a weighted chord vector;

[0030] Second, the weighted tone is marked and encoded to obtain a first mark vector of the weighted tone;

[0031] Finally, the note vector, the weighted chord vector, and the first mark vector are concatenated at corresponding positions, and the concatenated result is linearized, that is, the current note and the weighted chord are marked at the position of the weighted tone according to the first mark vector, to obtain an encoded guide vector;

[0032] The second feature encoding and concatenation unit is configured to encode the chord of the previous beat, the terminal chord, and the structural chord, and concatenate the encoded vectors to obtain a decoding guide vector, and comprises:

[0033] First, the chord of the previous beat is embedded and encoded to obtain a chord vector, and the structural chord in a historical period of time is embedded and encoded and then linearized to obtain a structural chord vector;

[0034] Second, the terminal chord is marked and encoded to obtain a second mark vector of the terminal chord;

[0035] Finally, the chord vector, the structure chord vector and the second mark vector are spliced at corresponding positions, and linearization processing is performed on the spliced result, that is, the last beat chord and the structure chord are marked with the second mark vector to obtain a decoding guide vector.

[0036] In an embodiment, in the chord generation module, when the Transformer model is used for chord generation, the input encoder guide vector is marked with the corresponding note position, and after waiting for a preset number of notes, the preset number of encoding guide vectors are spliced and input into the encoder of the Transformer model for chord arrangement, wherein the preset number of notes is preferably the number of notes included in each beat.

[0037] In an embodiment, in the chord generation module, when the Transformer model is used for chord generation, the decoder performs chord generation calculation on the input decoding guide vector and the arrangement chord output by the encoder to generate a chord, and the generated chord at the current time and the historical generated chord are arranged to form an arrangement chord sequence.

[0038] The arrangement chord sequence is used as a data source to construct a decoding guide vector for a next chord generation task.

[0039] In an embodiment, the device further comprises:

[0040] The performance module comprises a sound and / or a performance instrument, wherein the performance instrument is a keyboard instrument, including a piano, an electronic organ and an accordion, and a light source is attached to each note key of the keyboard of the performance instrument, and the light source is preferably a light-emitting diode.

[0041] The performance control module is configured to control the sound and / or the arrangement chord sequence output by the chord generation module.

[0042] In an embodiment, when the performance mode control module controls the performance of the arrangement chord sequence, the performance mode adopted includes a learning mode, a composition mode, a practice mode and a playback mode.

[0043] The learning mode refers to that during real-time performance, after each beat of melody is played, the light source on the note key of the performance instrument is turned on to indicate the chord constituent note, but the chord is not played through the sound.

[0044] The composition mode refers to that during real-time performance, after each beat of melody is played, the light source on the note key of the performance instrument is turned on to indicate the chord constituent note, and the chord is played through the sound.

[0045] The practice mode refers to playing only the melody in the first performance, not playing the chord progression, saving the complete music of the chord generated by the chord generation module, and in the practice, the light source on the note key of the performance instrument is turned on according to the music, but the music is not played through the sound, and the performer needs to play the music by himself;

[0046] The playback mode refers to playing the music after the performance, playing back the music recording, and adding the chords generated by the chord generation module in the chord performance mode, and the light source on the note key of the performance instrument is turned on at the same time.

[0047] In one embodiment, the device further comprises:

[0048] The visual editing module is used to set the performance mode, audio input mode, music tonality, and beat through the visual interface.

[0049] The preprocessing module is used to select the preprocessing mode corresponding to the audio input mode according to the set audio input mode, preprocess the input audio data to obtain the melody corresponding to the audio, and input the melody to the feature extraction module.

[0050] Compared with the prior art, the present application has at least the following beneficial effects:

[0051] The feature extraction module extracts four types of high-level features, including weight tone, weight chord, terminal chord, and structure chord, from the input melody, and the feature coding and splicing module constructs the coding guide vector and the decoding guide vector based on the high-level features, and uses the coding guide vector and the decoding guide vector to guide the generation of deeper chords by the Transformer model, so as to improve the accuracy and quality of the generated chords, and the performance instrument and the performance mode are also provided to display and apply the generated chords. BRIEF DESCRIPTION OF DRAWINGS

[0052] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0053] Figure 1 is a structural schematic diagram of the music creation auxiliary device provided by the embodiment;

[0054] Figure 2 is a generation schematic diagram of the coding guide vector provided by the embodiment;

[0055] Figure 3 is a generation schematic diagram of the decoding guide vector provided by the embodiment;

[0056] Figure 4 This is a schematic diagram of the Transformer model used in the embodiment to generate chords;

[0057] Figure 5 This is a diagram of the staff notation provided in the embodiment;

[0058] Figure 6 This is an application flowchart of the music creation aid device provided in the embodiment. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0060] To address the issue of low chord generation quality caused by ignoring higher-level musical features and inaccurate guiding conditions, the embodiments provide, for example... Figure 1 The music creation aid shown achieves high-quality and high-accuracy chord generation through the coordinated operation of a feature extraction module, a feature encoding and splicing module, and a chord generation module.

[0061] Feature extraction module

[0062] In this embodiment, to consider the high-level features of the music, the Transformer model needs to be guided by knowledge to make the generated chord music more accurate and pleasing to the ear. Based on this, the provided feature extraction module is used to extract high-level features of the input melody. Four musical features are proposed as high-level features based on Schenk theory and common musical sense: weighted notes, weighted chords, structural chords, and cadence chords. These four high-level features serve as conditions controlling the Transformer model's chord generation. Among them, weighted notes and weighted chords are melodic features, while structural chords and cadence chords are features of the chords generated by the Transformer model.

[0063] In this embodiment, some basic musical terms are first introduced:

[0064] 4 / 4 time signature: A common time signature in music, with a quarter note as one beat and four beats per measure.

[0065] The pattern of strong and weak notes: The intensity of each note varies depending on the beat.

[0066] Chord progressions: Two or more chords are played in chronological order. For example, in a V-I chord progression, the fifth chord is played first, followed by the first chord.

[0067] Mode: A system of notes organized logically and centered on a certain note. The nature and characteristics of the mode are called mode.

[0068] Chord progression: The order of the notes in the mode. The order number of the root note of the chord is the chord progression. For example, the first chord in C major is C chord, and the root note is Do. The fourth chord is F chord, and the root note is Fa.

[0069] When identifying the chord progression, refer to the tonic determined by the mode and the interval difference determined by the mode, determine the natural scale under the mode, and determine the relative position of the chord root note in the natural scale. The corresponding order number is the chord progression.

[0070] In the embodiment, the feature extraction module includes a weight note extraction unit, a weight chord extraction unit, a terminal chord extraction unit, and a structure chord extraction unit.

[0071] The weight note extraction unit is used to extract the rhythm accent, cut note and long note in the melody, and select the note that is a rhythm accent but not a cut note, or a note that is not a rhythm accent but is a cut note and a long note as a weight note. The weight note is decisive for music, and has a guiding role in generating accompaniment. Since the definition of the weight note is based on the weight note, the cut note and the long note, the weight note extraction includes:

[0072] First, the weight note T, the cut note C and the long note L are extracted by using formulas (1)-(3);

[0073] T=N strong ∪N semi-strong (1)

[0074]

[0075] L=N latest ∩N longest (3)

[0076] Wherein, N represents a note, L(N) represents the duration of a note, and the subscript of N represents the nature of the note, wherein, N strong represents a note at a rhythm strong beat, N semi-strong represents a note at a rhythm secondary strong beat, N weak represents a note at a rhythm weak beat, steps represents the distance from the starting position of the note to the next strong beat or secondary strong beat, represents a note with a duration greater than N latest represents the last long note, N longest represents the note with the longest duration.

[0077] Then, after extracting the weight notes, the cut notes and the long notes, the weight notes are analyzed in a sliding window manner, and the size of the window is set to the last 16 16th notes, i.e. 4 beats. In each sliding window, formula (4) is used to select the rhythm accent T but not the cut note or the rhythm accent but the note that is both the cut note C and the long note L as the weight note WN.

[0078]

[0079] In the embodiment, the weight chord extraction unit is configured to extract a chord composed of representative notes in the pre-sequence melody segment starting from the weight note as the weight chord. The weight chord is based on the weight note and represents the overall characteristics of the pre-sequence melody, and is specifically a chord composed of several representative notes in the pre-sequence melody.

[0080] In one embodiment, the extraction process of the weight chord includes:

[0081] (a) constructing a chord reference table containing a plurality of chords, wherein the chord reference table can contain 36*12=432 chords, which means that there are 36 chord combinations for each root note, corresponding to 36 attributes of each root note, and on this basis, 36*12=432 chords are composed by taking 12 notes in an octave as root notes respectively. The chord reference table is a template for comparison when determining the weight chord and the structure chord. In the embodiment, a hash table data structure can be used to store a large number of chord templates to improve the running speed.

[0082] (b) performing frequency statistics of the weighted pitch of all notes in the current melody segment starting from the weight note in a manner that the weight note is weighted by α and the other notes are weighted by β to obtain a first weighted statistical result of the current melody segment, wherein α>β, for example, α takes a value of 2 and β takes a value of 1.

[0083] (c) concatenating the current melody segment starting from the weight note with all melody segments in the adjacent historical group to form a melody sequence, and performing frequency statistics of the weighted pitch of all notes in the melody sequence in a manner that the weight note is weighted by α and the other notes are weighted by β to obtain a second weighted statistical result of the melody sequence, wherein α>β.

[0084] (d) comparing the first weighted statistical result and the second weighted statistical result with the chord reference table respectively, and selecting a chord with the minimum editing cost from the chord reference table as the weight chord corresponding to the weight note, wherein the editing cost refers to the operation complexity of converting the weighted statistical result into a chord in the chord reference table, and the chord with the minimum editing cost contains the representative notes in the current melody segment.

[0085] In the embodiment, the chord contrast table is used to select the chord with the minimum editing cost as the weight chord corresponding to the weight of the weight tone, including:

[0086] The first weighted statistical result is compared with the chord contrast table, and the chords in the chord contrast table are sorted in ascending order according to the editing cost to obtain a first chord sorting list, chords with roots being natural scales under the tonality of the current mode are selected as candidate chords from the first chord sorting list, and the candidate chord with the minimum editing cost is selected as a first candidate weight chord, wherein the natural scale is two notes apart on the keyboard.

[0087] The second weighted statistical result is compared with the chord contrast table, and the chords in the chord contrast table are sorted in ascending order according to the editing cost to obtain a second chord sorting list, chords with roots being natural scales under the tonality of the current mode are selected as candidate chords from the second chord sorting list, and the candidate chord with the minimum editing cost is selected as a second candidate weight chord.

[0088] The first candidate weight chord and the second candidate weight chord are selected from the first candidate weight chord and the second candidate weight chord, and the minimum editing cost is selected as the maximum weight chord.

[0089] When the maximum weight chord is selected from the first candidate weight, the current melody is segmented into a new group, and the new group is used as a neighboring historical group for weight chord calculation at the next time; when the maximum weight chord is selected from the second candidate weight, the current melody segment is divided into a neighboring historical group to update the neighboring historical group. It should be noted that the neighboring historical group contains a plurality of melody segments that are most adjacent in time to the current melody segment, and the melody segments belonging to the same neighboring historical group have the same weight chord.

[0090] In the embodiment, the termination chord extraction unit is used to identify the termination or semi-termination of the chord in progress to determine the termination chord. The termination chord is used to mark the termination of the chord in progress, which can be used to divide the sentence to make the chord structure more clear. The identification of the termination chord is to identify the termination and semi-termination of the chord in progress. The termination includes the following three types: first, the positive termination, Ⅴ-Ⅰ termination, that is, the chord progression from the fifth level to the first level, which is understood as the positive termination when the situation from the fifth level chord to the first level chord occurs; second, the variable termination, Ⅳ-Ⅰ termination, that is, the chord progression from the fourth level to the first level; third, the obstacle pause, that is, the Ⅴ-Ⅰ progression is replaced by Ⅴ-Ⅵ, that is, the chord progression from the fifth level to the first level is replaced by the chord progression from the fifth level to the sixth level. The semi-termination is any chord to the V level or VII level chord.

[0091] In one embodiment, the process of extracting the termination chord includes:

[0092] First, the terminal mode and semi-terminal mode are extracted according to formula (5) and (6);

[0093] terminal mode = Chord V-I ∪ Chord IV-I ∪ Chord V-VI sub V-I (5)

[0094] semi-terminal mode = Chord ～ -V / VII (6)

[0095] wherein Chord V-I represents a V-I cadence, i.e. a chord progression from the fifth to the first degree, Chord IV-I represents a IV-I cadence, i.e. a chord progression from the fourth to the first degree, Chord V-VI sub V-I represents a V-I progression replaced by a V-VI progression, i.e. a chord progression from the fifth to the first degree replaced by a chord progression from the fifth to the sixth degree, Chord ～ -V / VII represents a progression from any chord to a V or VII chord, i.e. a progression from any chord to a chord of the fifth or seventh degree.

[0096] Then, after obtaining the terminal mode and semi-terminal mode, the terminal chord TC is determined according to formula (7);

[0097] TC = Chord terminal ∪ Chord semi-terminal (7)

[0098] In the embodiment, when identifying the tonality and mode of a piece of music, the following scale numbers refer to the scale numbers in the tonality and mode. For semi-terminal mode: recognition of a progression of a VII chord (i.e. a chord of the seventh degree); for perfect cadence: recognition of a pattern of II / IV / VI - V - I (i.e. a chord of the second degree or a chord of the fourth degree or a chord of the sixth degree followed by a chord of the fifth degree followed by a chord of the first degree) for two consecutive chords; for imperfect cadence: recognition of a pattern of II / IV / VI - I (i.e. a chord of the second degree or a chord of the fourth degree or a chord of the sixth degree followed by a chord of the first degree) for two consecutive chords; for obstructive cadence: recognition of a pattern of V - VI (i.e. a chord of the fifth degree followed by a chord of the sixth degree) for two consecutive chords.

[0099] In the embodiment, the algorithmic process for determining the terminal chord includes: setting all variables is_cadence to 0, 0 representing a non-terminal chord and 1 representing a terminal chord, and performing the following steps at the end of each bar:

[0100] (a) Take the previous 16 consecutive chords sequence including the current chord as the analysis object, the chord sequence can be the chord sequence generated by the Transformer model when the Transformer model is applied in a loop;

[0101] (b) Use the above rules to identify the half-cadence, perfect-cadence, imperfect-cadence, and blocked-cadence;

[0102] (c) If the current chord matches one of the cadence patterns, identify the current chord as a cadence chord.

[0103] (d) Update the global variable is_cadence, if it is a cadence chord, update its value to 1; Note that if the last chord of the cadence continues, the corresponding is_cadence of each beat during the continuation is assigned a value of 1, otherwise is_cadence is 0, representing that no cadence chord is identified.

[0104] In the embodiment, the structural chord extraction unit is configured to extract non-inversion chords in the melody, and if the non-inversion chord is a I, II, IV or V degree chord of the current mode, the current non-inversion chord is a structural chord. Structural chords play an important role in maintaining the stability and harmony of the accompaniment before and after. The chord generated by the Transformer model must meet the following requirements to be judged as a structural chord. First, the structural chord is a non-inversion chord, that is, the root note is the lowest note. Further, the structural chord is a I, II, IV or V degree chord of the current mode. Therefore, the extraction formula of the structural chord SC is shown in formula (8):

[0105] SC = Chord non-inverted ∩(Chord Ⅰ ∪Chord Ⅱ ∪Chord Ⅳ ∪Chord Ⅴ ) (8)

[0106] Wherein, Chord non-inverted represents a non-inversion chord, Chord Ⅰ , Chord Ⅱ , Chord Ⅳ , Chord Ⅴ represent I, II, IV, V degree chords of the current mode respectively.

[0107] Specifically, a global list type variable structural_chord[] is defined, and the elements of the variable are chord constituent notes, generally 3-5 elements, and in rare cases 6-8 elements. The input is a chord pitch sequence curchord[] saved in the form of a list, where each element is an integer representing the pitch of the chord constituent note. The specific chord extraction process is as follows:

[0108] First, the input chord is compared with the chord reference table. If the input chord exists in the chord reference table, the input chord is taken as a candidate chord. For the first input, the input chord is the I-level chord under the current mode tonality, and in the training generation task, the input chord is the generated chord output by the Transformer model.

[0109] Then, it is judged whether the candidate chord is the I, II, IV or V-level chord under the current mode tonality. When it is the I, II, IV or V-level chord under the current mode tonality, the candidate chord is a structural chord.

[0110] Feature encoding and splicing module

[0111] In the embodiment, based on a strategy similar to the attention pre-control, the condition is input before the self-attention layer, and in order to make up for the insufficient condition guidance leading to poor quality chords generated by the Transformer model, a feature encoding and splicing module is introduced. The feature encoding and splicing module is used to encode the four types of high-level features of the weight note, the weight chord, the terminal chord and the structural chord extracted based on the feature extraction module, and the encoded vector is used as a condition to guide the knowledge of the Transformer model, so as to improve the quality of the chords generated by the Transformer model.

[0112] In the embodiment, the feature encoding and splicing module includes a first feature encoding and splicing unit and a second feature encoding and splicing unit. The first feature encoding and splicing unit is used to encode the current note, the weight note and the weight chord and splice the encoded vectors to obtain an encoded guidance vector, as shown in Figure 2 The specific structure includes:

[0113] First, the current note is embedded and encoded to obtain a note vector, and the weight chord is embedded and encoded and then linearly processed to obtain a weight chord vector. Second, the weight note is marked and encoded to obtain a first mark vector of the weight note. Finally, the note vector, the weight chord vector and the first mark vector are spliced at corresponding positions, and the spliced result is linearly processed, that is, the current note and the weight chord are marked with the position of the weight note according to the first mark vector, to obtain an encoded guidance vector. The formula is:

[0114]

[0115]

[0116] wherein WN represents a weight note, flag1 represents a position marker result of the weight note, when the note is a weight note, the weight note is marked as 1, otherwise marked as 0, Emb(note) represents an encoding vector of the note note, Emb(WF) represents an encoding vector of the weight chord WF, Linear(·) represents a linear layer mapping processing, represents a splicing operation, Output en represents a decoding guide vector, the Output en is used for conditional constraint in the encoding stage of the Transformer model.

[0117] The second feature encoding and splicing unit is used for encoding and splicing the encoding vectors of the chord of the last beat, the final chord and the structure chord to obtain a decoding guide vector, as shown in Figure 3 , specifically comprising:

[0118] First, the chord of the last beat is embedded and encoded to obtain a chord vector, and the structure chord of a historical period of time is embedded and encoded and then linearly processed to obtain a structure chord vector; second, the final chord is marked and encoded to obtain a second marker vector of the final chord; finally, the chord vector, the structure chord vector and the second marker vector are spliced in corresponding positions, and the spliced result is linearly processed, that is, the chord of the last beat and the structure chord are marked in the position of the final chord according to the second marker vector to obtain a decoding guide vector, which is expressed by a formula as:

[0119]

[0120]

[0121] wherein TC represents a final chord, flag represents a position marker result of the final chord, when the chord is a final chord, the final chord is marked as 1, otherwise marked as 0, Emb(chord) represents an encoding vector of the chord chord, Emb(SC) represents an encoding vector of the structure chord SC, Linear(·) represents a linear layer mapping processing, represents a splicing operation, Output dn represents a decoding guide vector, the Output dn is used for conditional constraint in the decoding stage of the Transformer model.

[0122] Chord generation module

[0123] In the embodiment, the chord generation module is configured to input the encoded guide vector as an input of an encoder of the Transformer model, to decode the guide vector as an input of a decoder of the Transformer model, and to perform a forward conduction operation on the Transformer model to generate the chord.

[0124] In the embodiment, the chord generation method using the Transformer model is creatively improved in input and output, that is, different from the rule-based model which makes constraints in the generation stage, the embodiment performs feature processing at the input layer of the encoder and the decoder of the Transformer model as a preset condition to guide the model to generate the chord. In this way, chords that are more in line with music theory, more harmonious with melodies, and more pleasant to listen to can be created.

[0125] Since the music creation assistance device provided by the embodiment is automatic chord generation in real time, the Transformer model can only see the input before the current time. As time goes on, the structure processed by the Transformer model will become longer and longer, and the processing time will also become larger and larger, which is intolerable in real-time tasks. To solve this problem, the input length of the music sequence is limited, but this will also cause the loss of long sequence information, resulting in inaccurate prediction. To solve this problem, the embodiment extracts four types of high-level features, namely, weighted notes, weighted chords, terminal chords, and structural chords from long-time melodies as global information, which cleverly solves the problem of limited input sequence length and also ensures the global information of long sequence information, improving the accuracy of chord prediction.

[0126] In the embodiment, in order to improve the speed of the Transformer model, the processing length of the Transformer model is specified to be at least one beat, for example, one beat. Although this will cause the Transformer model to ignore the complete input sequence, the weighted chords and structural chords extracted from a long melody can effectively avoid this problem. In addition, in order to ensure that the Transformer model can obtain sufficient information, a "wait for a period of time k" (wait-k) strategy is also proposed. Instead of encoding once for every input sampling information, it waits for N times of sampling, for example, 4 times of sampling, that is, one beat, to perform encoding. In this way, the Transformer model can perform chord encoding with high accuracy and low latency.

[0127] Specifically, as shown in FIG. 6, the chord generation module 100 is configured to input the encoded guide vector as an input of an encoder of the Transformer model, to decode the guide vector as an input of a decoder of the Transformer model, and to perform a forward conduction operation on the Transformer model to generate the chord. Figure 4As shown in the chord generation module, when the Transformer model is used for chord generation, the position label of the corresponding note is marked on the input encoder (Transformer Encoder) coding guide vector, and after waiting for a preset number of notes (i.e., N times), the preset number of coding guide vectors are spliced and input into the encoder of the Transformer model for chord arrangement, wherein the preset number of notes is preferably the number of notes included in each beat, preferably 4 notes, i.e., sampling 4 times to obtain a coding guide vector for one beat for chord arrangement.

[0128] As shown in Figure 4 As shown in the chord generation module, when the Transformer model is used for chord generation, the position label of the corresponding note is marked on the input encoder (Transformer Encoder) coding guide vector, and after waiting for a preset number of notes (i.e., N times), the preset number of coding guide vectors are spliced and input into the encoder of the Transformer model for chord arrangement, wherein the preset number of notes is preferably the number of notes included in each beat, preferably 4 notes, i.e., sampling 4 times to obtain a coding guide vector for one beat for chord arrangement.

[0129] The Transformer model is stacked by a plurality of successive self-attention layers, and when applied to the real-time chord generation task, it can capture key information in the music data. The Transformer model directly processes music feature information after input and output modification, and can generate music chords with high accuracy. Through distillation, the processing time of the Transformer model is shortened to about 0.17s, which is sufficient to support real-time chord generation. The real-time generated staff chord information is saved in an array and output to the corresponding hardware.

[0130] It should be noted that before the Transformer model is applied to chord generation, it needs to be trained to optimize the model parameters. In the training process, in order to simulate the real-time input of music, a streaming processing method is creatively adopted, i.e., the melody information is input into the Transformer model in sequence, rather than all at once, to simulate a real-time scenario. The processing and construction of the data set used in the training process are described in detail in the subsequent data set construction stage.

[0131] Performance module and control module

[0132] The music creation assistance device provided by the embodiment further comprises a performance module and a performance control module, wherein the performance module comprises a sound box and / or a performance instrument, wherein the performance instrument is a keyboard instrument, including a piano, an electronic organ, and an accordion, and a light source is attached to each note key of the keyboard of the performance instrument, and the light source is preferably a patch light-emitting diode (LED). The performance control module is used to control the sound box and / or the performance of the arranged chord sequence output by the chord generation module.

[0133] In the embodiment, the sound is used to play the program predicted harmony, or to play the playback of the chord performance. In terms of harmony performance, the column chord is designed as the form of accompaniment playing, and the system will play accompaniment according to the generated chord. The chord performance mode can be described in the program as follows: [start time (beat), duration (beat)]. The brief example is as follows: [0, 2], [2, 1], [3, 1]. For example, when the played chord is C major C major triad, the actual performance result of the above example is as shown in the following staff format. According to this format, four different rhythm chord accompaniments are set in the device: Figure 5

[0134] 1. [0, 4]

[0135] 2. [0, 2], [2, 2]

[0136] 3. [0, 2], [2, 1], [3, 1]

[0137] 4. [0, 1], [1, 1], [2, 1], [3, 1]

[0138] The program inside will convert the chord to be played according to the above rhythm control format into midi instructions, which can be output into a midi format file through midi file or directly played through the tiny sound font sound source library.

[0139] The patch LED refers to a group of LED lights that emit light and extinguish under the control of the performance control module according to the set order and time, forming a certain visual effect. In the embodiment, the LED light is used to indicate the current chord pitch sequence generated by the model, so that the performer can know the generated chord in a conspicuous and convenient way and play or create according to the prompt.

[0140] In actual use, sixty-one independent LED lights can be used to represent twelve notes of five octaves in the current tonality mode, which is also compatible with the standard sixty-one key electronic piano key position. The LED light can be manually pasted above the key, which has good prompt effect and low cost, solves the compatibility problem of different intervals and different number of keys of the piano, thereby expanding the use of the audience and improving the experience of the performer.

[0141] The performance control module inside uses an array to store the marks of the light and dark of different LED lights. When receiving the current beat chord transmitted by the model, since the chord value transmitted by the model is between 36 and 96, which corresponds to sixty-one keys, the on-off state of the LED light can be easily calculated according to the chord pitch, and the corresponding signal is transmitted to the hardware logic, that is, the control of the LED light is completed.

[0142] ​In the embodiment, the output device can be divided into four modes according to the performance in the performance and after the performance: learning mode, creation mode, playback mode and practice mode. The performance control module controls the performance of the chord sequence in the above four modes. The four design modes ingeniously solve the different problems that musicians are troubled in the four scenes, and are highly adaptable to groups with different needs and have a wide range of audience.

[0143] The learning mode refers to that during the real-time performance, after playing a beat of melody, the light source on the note key of the performance instrument is turned on to indicate the chord composition, but the chord is not played through the sound. In this learning mode, the performer can clearly understand the chord just played through the turned-on LED light and follow the performance, use the model prediction result as a reference, and continuously deepen the understanding of the melody and the analysis of the accompaniment. At the same time, the performer can more intuitively feel the common chord progression through the indication of the light, so as to make the performance skill more skilled and improve the familiarity with the improvisation accompaniment skill.

[0144] The composition mode refers to that during the real-time performance, after playing a beat of melody, the light source on the note key of the performance instrument is turned on to indicate the chord composition, and the chord is played through the sound. Since more attention is paid to the main melody when arranging music, in this mode, the composer can understand the chord corresponding to the melody just played through the sound effect, so as to guide the creation of the subsequent melody. The turning-on of the LED light can assist the composer in deconstructing the chord, help him more fully and reasonably use the arrangement skill, make the music flow smoothly, and make the creation process more natural and easy, so as to achieve the purpose of assisting the creation.

[0145] The practice mode refers to that in the first performance, only the melody is played, the chord progression is not played, the staff chord generated by the chord generation module is saved in the complete music, in the practice, the light source on the note key of the performance instrument is turned on according to the music, but the music is not played through the sound, and the performer needs to play the music by himself. In this mode, the sequence information of the chord and the melody has been stored in advance, so that the real-time display can be realized when the music is played again. The performer can play the chord and the melody by following the LED light, so that the performance of the whole music can be more skilled and the understanding can be more profound.

[0146] The playback mode refers to that after the performance of the music, the music recording is played back, and the chord generated by the chord generation module is added in the staff performance mode, and the light source on the note key of the performance instrument is turned on at the same time. This can help the performer clearly and intuitively observe the music, on the one hand, the performer can appreciate the just played music, on the other hand, the performer can also be inspired to modify and further create.

[0147] The visual editing module and the input mode selection and preprocessing module

[0148] In the embodiment, the visual editing module is configured to set the performance mode, the audio input mode, the music key, and the beat through the visual interface; and the preprocessing module is configured to select a preprocessing mode corresponding to the audio input mode according to the set audio input mode, preprocess the input audio data to obtain a melody corresponding to the audio, and input the melody to the feature extraction module.

[0149] In the application, the user sets the performance mode, the audio input mode, the music key, and the beat through the visual interface provided by the visual editing module.

[0150] For the microphone input, the microphone collects audio data and inputs the audio data to the preprocessing module.

[0151] First, the zero-crossing rate of each piece of audio data is calculated, and whether the piano sound is input is determined according to the zero-crossing rate, that is, if the zero-crossing rate is greater than a set threshold, it is considered that no piano sound is input, otherwise, the pitch recognition is performed.

[0152] The main application scenario of the microphone is the ordinary piano and other instruments without midi input conditions, so that the conversion process from the physical signal to the digital signal can be effectively and relatively accurately completed, and the performer can easily use the music creation auxiliary device without the midi input conditions.

[0153] For midi input, for devices with standard midi interface, such as electronic organ, organ, hand roll piano, etc., midi signal can be directly used for input, i.e. the collected midi signal is input to the preprocessing module, and then the audio data is preprocessed by using the preprocessing mode corresponding to the midi keyboard input mode to obtain the pitch sequence for chord prediction. The preprocessing of the midi signal includes: according to the current beat speed BPM, receiving the pitch input of the midi signal, sampling every sixteenth note, and setting a buffer to store four sixteenth notes (i.e. one beat). When the buffer is full of four sixteenth notes, the pitch sequence of the current beat is sent to the feature extraction module for high pitch feature extraction, while the next beat data is continuously read.

[0154] The advantage of midi signal input is mainly higher accuracy and lower signal processing complexity, and there are corresponding input signal methods for different types of musical instruments.

[0155] In the embodiment, the above-mentioned feature extraction module, feature encoding and splicing module, chord generation module, performance control module and integration in the chip can adopt ASIC chip, which is used to run chord generation algorithm and control the overall process. For the chip, in addition to performance, the indicators to be considered also include isomorphism, cost and power consumption, etc., therefore, the ASIC chip generated by Cambrian, Horizon and other companies can be preferred. The pre-processing of input audio data, high-level feature extraction, construction of encoding condition vector and decoding condition vector, chord generation according to the encoding condition vector and the decoding condition vector, display output of the generated chord according to the playing mode control, i.e. control of the output of the patch LED and the sound of the chord. Specifically, the process of music creation by applying the music creation auxiliary device provided by the embodiment is as shown in Figure 6 For melody information extraction, the melody in the environment can be collected by microphone for signal conversion, or the pitch information can be directly input by midi keyboard. After chord generation, the melody and chord sequence can be replayed after the end of the performance to practice and modify, giving users more vivid auditory and visual auxiliary effects.

[0156] The music creation assistance device provided by the embodiment can help musicians complete musical ideas and reduce stalls. According to the prompt of the metronome, the system generates the chord corresponding to the melody of the last beat after a short period of time after the musician plays a beat of melody with the right hand. The LED lights pasted on the corresponding position of the keyboard will light up partially to indicate the constituent notes of the chord. The performer can then follow the instructions while the lights are on, or use the sound playback columnar chord according to different selection modes. The creator will obtain the chord prompt corresponding to the melody just created after creating a beat of melody during the creation process, which helps the creator to better conceive and perform.

[0157] However, it should be noted that the chord generated by this mode is not a complete real-time accompaniment, but the chord of the last beat is generated after playing a beat, and its position in the melody should be at the beginning of the last beat. Therefore, in real-time tasks, the main purpose of the generated chord is to help the composer to assist in creation, to appropriately reduce the difficulty of chord arrangement, and to combine the design of the LED light. The creator receives clear and accurate prompt information, which can be used to a certain extent for composition work. For learners, real-time generation of chord information of melody is a good way to practice their harmony learning. Learners can compare the chord information generated by the model with their own ideas to improve and progress.

[0158] The music creation assistance device provided by the embodiment extracts high-level features of music using a music analysis method to solve the problem of neglect. Meanwhile, the Transformer model is further improved by adding constraints to the input stage of the encoder and decoder to achieve deeper music guided generation, which further improves the quality and accuracy of music. Finally, the software implementation of the entire creation process is combined with the corresponding hardware to realize the functions of the entire device.

[0159] Data set construction stage

[0160] In the embodiment, the construction process of the data set includes:

[0161] 1. Obtain music files saved in XML format, and the number of music scores is 15000. The XML file includes header information and main data, wherein the main data includes melody part and harmony part. The header information is shown in Table T1:

[0162] Table T1 Header information table

[0163]

[0164] The header information table includes the overall information of the song, specifically: artist <artist>, title of a musical piece <title>, number of beats in measure <beats_in_measure>, tempo< / title> <bpm>, tonality <key>, mode <mode>.

[0165] The melody part in the main data is shown in Table T2:

[0166] Table T2 Main data (melody part)

[0167]

[0168]

[0169] The melody part contains the melody information of the song, consisting of a number of notes <note>The note is composed of: the starting beat of the note in the whole song <start_beat_abs>, the measure the note belongs to in the whole song <start_measure>, the starting beat of the note in the current measure <start_beat>, the note length <note_length>, the scale degree <scale_degree>, the octave <octave> <octave>, whether or not in repose <isrest>.

[0170] The harmony part in the main data is shown in Table T3:

[0171] Table T3 Main data (harmony part) table

[0172]

[0173]

[0174] The harmony part contains the harmony information of the song, which is composed of several chords <chord>Construction. Specifically: chord progression <sd>, and chord type <fb> <sec> <sus> <pedal> <alternate> <borrowed>chord duration, start measure, start beat, start beat abs, rest <isrest>.

[0175] 2. Remove unusable data from the dataset. The dataset needs to exclude data with a large number of empty notes or incomplete chord sequences. Such data needs to be treated as dirty data and screened and cleaned; otherwise, it will lead to a mismatch between melody and harmony, significantly impacting the model.

[0176] 3. Only retain data with 4 beats per measure. Because different time numbers can cause ambiguity when the time dimension is removed and the data is reduced to a single digit, only data with 4 beats per measure is retained, i.e., data in the header information.<beats_in_measure> The data has a value of 4.

[0177] 4. Remove all keys except major and minor. Popular music is mostly in major or minor keys, while chord progressions in medieval and other modes are more difficult to predict. Therefore, only the information in the XML file header is retained. <mode>Data of values 1 or 6 (1 for major, 6 for minor). After the above deletion process, there are about 6000 pieces of musical data in total.

[0178] 5. Conversion to pitch and duration sequence storage. Read the XML file in <data>The data is partially parsed into melody sequence and chord sequence. Each sequence is a sequence of pitch-time pairs. The melody sequence is a sequence of single notes, and the chord sequence is a sequence of multi-note sets. The pitch is annotated by the standard Midi pitch (Midi absolute pitch), and the time is calculated by 1.0 per beat and 4.0 per bar.

[0179] First, the header information is determined, and the data is parsed into a melody sequence and a chord sequence. Each sequence is a sequence of pitch-time pairs. The melody sequence is a sequence of single notes, and the chord sequence is a sequence of multi-note sets. The pitch is annotated by the standard Midi pitch (Midi absolute pitch), and the time is calculated by 1.0 per beat and 4.0 per bar. <key>and <mode>The tag determines the musical key.

[0180] The melody sequence note translation process is as follows:

[0181] For a single note, the note scale degree is determined according to the <scale_degree>, the note pitch is determined according to the musical key previously determined by the header information, and finally the note is translated according to the <note_translations> table. <octave>tag, determine if the scale degree is transposed by an octave. For example: the music key is A major, the scale_degree is 1, <octave>If the value of the variable is 0, then this note is the first scale degree in the key of A major, i.e. A, which is 6 in the simplified notation. Also, because <octave>is 0, so this pitch is A4 in scientific pitch notation, so it corresponds to 57 in the midi standard pitch (Midi absolute pitch).

[0182] The chord sequence note transformation process is as follows:

[0183] For a single note, according to the <scale_degree> to determine the chord degree, and then according to the music tonality determined by the header information before, to determine the chord pitch, and finally the chord type, to determine the chord sound composition. For example: the music tonality is A major, the chord <sd>If the tag is 1 and the chord type tag is the default value, then this chord is a major triad in the key of A major, the root is A3, the third is a major third from the root, the third is C#4, the third and fifth are a minor third, the fifth is E4, so the corresponding midi standard pitch (Midi absolute pitch) is [45, 49, 52].

[0184] The <chord_duration> and <note_length> values are directly extracted from the chord and note time values. The time value sequence and the pitch sequence elements correspond one by one.

[0185] The converted partial data sample is shown in Table T4:

[0186] Table T4: Converted partial data sample table

[0187]

[0188] 6. Convert the key of all data to C major or A minor. The pitches of the same level numbers in different keys are different, and cannot be used to train the same model. However, the data between the relative keys is universal, such as C major and A minor. Therefore, all the music data is calculated by the functions of Tables T5 and T6 according to the key <key>The value is added or subtracted to all pitches of the melody sequence and the chord sequence to realize the transformation in the mode; all major songs are converted to C major, and all minor songs are converted to A minor.

[0189] Table T5: calculation function table 1

[0190]

[0191]

[0192] Table T6: calculation function table 2

[0193]

[0194] The results of the original data sample after the above function conversion are shown in Table T7:

[0195] Table T7: original data sample table after conversion

[0196]

[0197]

[0198] It can be observed that the number of notes and the duration do not change, and only the pitch value increases or decreases, representing that the original song is converted to C major.

[0199] 7. Remove the time dimension and convert the data dimension to one dimension.

[0200] Each note is sampled by duration, and the main melody sequence is sampled by sixteenth note, and the chord sequence is sampled by quarter note. The results of the one-dimensional data after the sample data conversion in step 6 are shown in Table T8:

[0201] Table T8: one-dimensional sample data table

[0202]

[0203] Finally, the data of each song becomes two sequences, and the task is converted to a sequence-to-sequence problem, which can be verified by using natural language processing models and methods.

[0204] The above specific embodiments have described the technical solutions and beneficial effects of the present application in detail. It should be understood that the above description is only the most preferred embodiment of the present application, and is not used to limit the present application. Any modification, supplement and equivalent replacement within the principle range of the present application should be included in the protection scope of the present application.< / key> < / sd> < / octave> < / octave> < / octave> < / mode> < / key> < / data> < / mode> < / isrest> < / borrowed> < / alternate> < / pedal> < / sus> < / sec> < / fb> < / sd> < / chord> < / isrest> < / octave> < / note> < / mode> < / key> < / bpm> < / artist>

Claims

1. A music composition assisting apparatus characterized by comprising: The method comprises the following steps: The feature extraction module is used to extract high-level features of the input melody, and the high-level features include weight notes, weight chords, terminal chords and structural chords; The feature encoding and splicing module is used to encode the current notes, weight notes and weight chords, splice the encoding vectors to obtain an encoding guide vector, and encode the chords of the previous beat, terminal chords and structural chords, and splice the encoding vectors to obtain a decoding guide vector; Specifically, the feature encoding and splicing module comprises: The first feature encoding and splicing unit is used to encode the current notes, weight notes and weight chords, and splice the encoding vectors to obtain an encoding guide vector, and comprises: Firstly, the current notes are embedded and encoded to obtain a note vector, and the weight chords are embedded and encoded and then linearly processed to obtain a weight chord vector; Secondly, the weight notes are marked and encoded to obtain a first mark vector of the weight notes; Finally, the note vector, the weight chord vector and the first mark vector are spliced at corresponding positions, and the spliced result is linearly processed, that is, the current notes and weight chords are marked by weight note positions according to the first mark vector, to obtain the encoding guide vector; The second feature encoding and splicing unit is used to encode the chords of the previous beat, terminal chords and structural chords, and splice the encoding vectors to obtain a decoding guide vector, and comprises: Firstly, the chords of the previous beat are embedded and encoded to obtain a chord vector, and the structural chords in a historical period of time are embedded and encoded and then linearly processed to obtain a structural chord vector; Secondly, the terminal chords are marked and encoded to obtain a second mark vector of the terminal chords; Finally, the chord vector, the structural chord vector and the second mark vector are spliced at corresponding positions, and the spliced result is linearly processed, that is, the chords of the previous beat and the structural chords are marked by terminal chord positions according to the second mark vector, to obtain the decoding guide vector; The chord generation module is used to take the encoding guide vector as the input of the encoder of the Transformer model, take the decoding guide vector as the input of the decoder of the Transformer model, and generate chords through forward conduction operation of the Transformer model.

2. The music composition assist device according to claim 1, characterized by The feature extraction module comprises: The weight note extraction unit is used to extract rhythm accents, cut notes and sustained notes in the melody, and select notes that are rhythm accents but not cut notes, or notes that are not rhythm accents but are cut notes and sustained notes as weight notes; The weight chord extraction unit is used to extract a chord composed of representative notes in a current melody segment starting from the weight notes as a weight chord; The terminal chord extraction unit is used to identify the terminal type or semi-terminal type of the chord in progress to determine the terminal chord; The structural chord extraction unit is used to extract non-transposed chords in the melody, and if the non-transposed chord is a I, II, IV or V chord of the current mode, the current non-transposed chord is a structural chord.

3. The music composition aid of claim 1, wherein, In the weight chord extraction unit of the feature extraction module, the extraction process of the weight chord comprises: (a) constructing a chord contrast table comprising a plurality of chords; (b) performing a frequency count of the pitches of all the notes in the current melody segment starting from the weight tone, weighted by the weight of the weight tone and by the weights of the other notes, to obtain a first weighted statistical result of the current melody segment, wherein , and , and >

0. ; (c) concatenating the current melody segment starting with the weight tone with all melody segments in the adjacent history group to form a melody sequence, and performing a pitch frequency statistics on all notes in the melody sequence in a manner that the weight tone is weighted with a weight , and other notes are weighted with a weight , to obtain a second weighted statistics result of the melody sequence, wherein > 0 ; (d) comparing the first weighted statistical result and the second weighted statistical result with the chord contrast table respectively, and selecting a chord with the minimum editing cost from the chord contrast table as the weight chord corresponding to the weight tone, wherein the editing cost refers to the operation complexity of converting the weighted statistical result into a chord in the chord contrast table, and the chord with the minimum editing cost contains notes that are representative in the current melody segment.

4. The music composition aid of claim 1, wherein, The chord with the minimum editing cost is selected from the chord contrast table as the weight chord corresponding to the weight tone, comprising: comparing the first weighted statistical result with the chord contrast table, and sorting the chords in the chord contrast table in ascending order according to the editing cost to obtain a first chord sorting list, selecting chords with root notes being natural scales under the current mode tonality as candidate chords from the first chord sorting list, and selecting a candidate chord with the minimum editing cost as a first candidate weight chord from the candidate chords; comparing the second weighted statistical result with the chord contrast table, and sorting the chords in the chord contrast table in ascending order according to the editing cost to obtain a second chord sorting list, selecting chords with root notes being natural scales under the current mode tonality as candidate chords from the second chord sorting list, and selecting a candidate chord with the minimum editing cost as a second candidate weight chord from the candidate chords; selecting the minimum editing cost from the first candidate weight chord and the second candidate weight chord as the final weight chord; when the final weight chord is selected from the first candidate weight, the current melody segment is segmented into a new group, and the new group is used as the adjacent historical group for the next time weight chord calculation; when the final weight chord is selected from the second candidate weight, the current melody segment is segmented into the adjacent historical group to update the adjacent historical group.

5. The music composition aid of claim 1, wherein, In the chord generation module, when the Transformer model is used for chord generation, the position markers of corresponding notes are added to the encoding guide vectors input to the encoder, and after a preset number of notes are input, the encoding guide vectors of the preset number of notes are spliced and input to the encoder of the Transformer model for chord composition.

6. The music composition aid of claim 1, wherein, In the chord generation module, when the Transformer model is used for chord generation, the decoder performs chord generation calculation on the input decoding guide vector and the output composition chord of the encoder to generate a chord, and the generated chord at the current time and the historical generated chord are arranged to form an arrangement chord sequence. The arrangement chord sequence is used as a data source to construct the decoding guide vector of the next chord generation task.

7. The music composition aid of claim 1, wherein, The device further comprises: a performance module, the performance module comprising a sound box and / or a performance instrument, wherein the performance instrument is a keyboard instrument, including a piano, an electronic organ, and an accordion, and a light source is attached to each note key of the keyboard of the performance instrument, and the light source is a light-emitting diode; a performance control module for controlling the sound box and / or the arrangement chord sequence output by the chord generation module.

8. The music composition aid of claim 7, wherein, The performance control module adopts a performance mode including a learning mode, a composition mode, a practice mode, and a playback mode when controlling the performance of the arrangement chord sequence. The learning mode refers to that in real-time performance, after playing one beat of melody, the light source on the note key of the performance instrument is turned on to indicate the chord composition, but the chord is not played through sound; The composing mode refers to that in real-time performance, after playing one beat of melody, the light source on the note key of the performance instrument is turned on to indicate the chord composition, and the chord is played through sound; The practicing mode refers to that in the first performance, only the melody is played, the chord is not played, the complete music of the staff chord generated by the chord generation module is saved, in the practice, the light source on the note key of the performance instrument is turned on according to the music, but the music is not played through sound, and the performer needs to play the music by himself; The replay mode refers to that after the performance of the music, the music recording is played back, and the chord generated by the chord generation module is added in the chord performance mode, and the light source on the note key of the performance instrument is turned on at the same time.

9. The musical composition assistance device according to claim 1 or 7 or 8, characterized by, The device further comprises: A visual editing module for setting the performance mode, the audio input mode, the music tonality and the beat through a visual interface; A preprocessing module for selecting a preprocessing mode corresponding to the audio input mode according to the set audio input mode, preprocessing the input audio data to obtain the melody corresponding to the audio, and inputting the melody to the feature extraction module.

Images