Information processing method, device, electronic device and storage medium

Through coding processing and neural network generation model based on music theory strategies, the problem of insufficient diversity and innovation in melody generation in the existing technology is solved, and the automatic generation of high-quality melody is realized. Combined with music theory rules and deep learning technology, the generated melody is achieved. Have long-range relationships and composition techniques.

CN113920969BActive Publication Date: 2024-11-26BEIJING LINGDONGYIN TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202111178341.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-10-09
Publication Date
2024-11-26
Estimated Expiration
2041-10-09

AI Technical Summary

Technical Problem

The prior art is difficult to generate high-quality melody, especially the method of automatically generating melody based on encoding has insufficient diversity and innovation, and complex musical rules or evaluation functions are often required.

Method used

Through the pre-configured music theory strategy, the melody information to be processed and the chord information to be matched are encoded. The neural network generates the model to output melody matching the chords, and combines the music theory rules and deep learning technology to generate long-range relationships and composition techniques. melody.

Benefits of technology

Achieve high-quality melody generation without writing complex rules, with diversity and innovation, style and emotion controlled by chords, and the quality of the melody generated is better than that of melody without chord constraints.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113920969B_ABST
    Figure CN113920969B_ABST
Patent Text Reader

Abstract

The present application discloses an information processing method, device, electronic device and storage medium. The specific implementation scheme is as follows: according to a preconfigured music theory strategy, the melody information to be processed and the chord information to be matched are encoded to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning of the melody of the melody information to be processed; according to the encoding result and the melody generation model, the target melody information corresponding to the chord information to be matched is obtained. By adopting the present application, high-quality melodies can be automatically generated based on encoding.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to an information processing method, device, electronic device and storage medium. Background Art

[0002] Since the information revolution, the way music and multimedia are disseminated has changed dramatically in a short period of time. This qualitative change has led to an explosive growth in the market demand for all kinds of music: whether it is singles, albums, MVs, karaoke with pop music or artistic creation as the main elements, or short videos, advertisements, animations, promotional films and film and television works that use music as auxiliary, or radio stations, anchors, and public space music that use music as background content, a large amount of original music is needed. How to automatically generate high-quality melodies based on encoding has become a technical problem that needs to be solved in this field. Summary of the invention

[0003] The present application provides an information processing method, device, electronic device and storage medium.

[0004] According to one aspect of the present application, there is provided an information processing method, comprising:

[0005] According to a preconfigured music theory strategy, the melody information to be processed and the chord information to be matched are encoded to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed;

[0006] According to the encoding result and the melody generation model, target melody information corresponding to the chord information to be matched is obtained.

[0007] According to another aspect of the present application, there is provided an information processing device, including:

[0008] The encoding module is used to encode the melody information to be processed and the chord information to be matched according to a preconfigured music theory strategy to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed;

[0009] The melody generation module is used to obtain target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model.

[0010] According to another aspect of the present application, there is provided an electronic device, including:

[0011] at least one processor; and

[0012] a memory communicatively connected to the at least one processor; wherein,

[0013] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method provided by any embodiment of the present application.

[0014] According to another aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method provided by any embodiment of the present application.

[0015] By adopting the present application, the melody information to be processed and the chord information to be matched can be encoded according to a preconfigured music theory strategy to obtain an encoding result, wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed, so that the target melody information corresponding to the chord information to be matched can be obtained according to the encoding result and the melody generation model, and the target melody information is a high-quality melody automatically generated based on the encoding.

[0016] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present application.

[0018] Figure 1 is a flowchart of an information processing method according to an embodiment of the present application;

[0019] Figure 2 is a schematic diagram of the structure of an information processing device according to an embodiment of the present application;

[0020] Figure 3 It is a block diagram of an electronic device used to implement the information processing method of an embodiment of the present application. DETAILED DESCRIPTION

[0021] The following is a description of exemplary embodiments of the present application in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0022] The term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The term "at least one" in this article means any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can mean including any one or more elements selected from the set consisting of A, B, and C. The terms "first" and "second" in this article refer to multiple similar technical terms and distinguish them. They do not mean to limit the order or to limit them to only two. For example, the first feature and the second feature refer to two types / two features. The first feature can be one or more, and the second feature can also be one or more.

[0023] In addition, in order to better illustrate the present application, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present application can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present application.

[0024] Computer automatic melody generation technology refers to the process of automatically generating melodies using algorithms by representing music scores in discrete numerical form, taking chord progression, speed, type, etc. as conditions. For this technology, existing technical solutions mainly include: 1) generating or splicing melodies based on melody libraries and music rules; 2) generating melodies using search algorithms based on evaluation functions, including heuristic searches or genetic algorithms; 3) generating melody sequences using neural networks, mostly by using RNN or LSTM models to predict the next note to complete melody generation.

[0025] For the above-mentioned 1st type of solution, templated melodies can be generated, but they are limited by the limited melody library and rules, and the diversity and innovation are poor, and the music rules are difficult to summarize and exhaust. For the above-mentioned 2nd type of solution, it is difficult to determine whether the evaluation function is effective. Although it is possible to generate melodies that do not rely on the existing melody library, the quality is poor; for the above-mentioned 3rd type of solution, it is possible to expect the model to generate better melodies through training without writing a large number of rules, but the LSTM models used in most of them are difficult to learn the logic of the melody in the long term, making the generated melody more random, and it is difficult to use techniques such as modulation between phrases. In terms of thematicity, it is not as good as the rule-based melody generation method.

[0026] By adopting the present application, features and codes can be extracted from melodies and chords based on simple music theory rules, and used as inputs of neural networks, so as to output a melody that matches the chord progression. Among them, the melody obtained by the output is generated based on the total length of the chord and the melody specified by the user and the beginning of the melody, and is a melody that matches the chord. By adopting the present application, not only can the writing of complex music rules or evaluation functions be avoided, but the generated melody also has certain long-range relationships and composition techniques, and the melody is of high quality and innovative. Since the generation and processing of the melody of the chord progression specified by the user is realized, a melody that matches the chord progression is generated, and since the melody is subject to the given chord constraint, the melody is better than without the chord constraint, and the style and emotion can be controlled by the chord; multiple generation under the same conditions has a certain diversity, and certain composition techniques can appear.

[0027] The embodiment of the present application provides an information processing method. Figure 1 It is a flowchart of an information processing method according to an embodiment of the present application. The method can be applied to an information processing device. For example, the device can be deployed in a terminal or a server or other processing device to perform encoding processing, melody generation, etc. The terminal can be a user equipment (UE), a mobile device, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. In some possible implementations, the method can also be implemented by a processor calling a computer-readable instruction stored in a memory. For example Figure 1 As shown, including:

[0028] S101. According to a preconfigured music theory strategy, encode the melody information to be processed and the chord information to be matched to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed.

[0029] S102: Obtain target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model.

[0030] In an example of S101-S102, the target melody information is constrained by the given chord to be matched. For example, the target melody information can be generated based on the chord information to be matched given by the user, the total melody length of the melody information to be processed, and the beginning part of the melody of the melody information to be processed. Thus, the target melody information corresponds to the chord information to be matched given by the user, and the target melody information is better than that without the constraint of the chord information to be matched, and the style and emotion of the target melody information can be randomly controlled by the chord information to be matched. Then, with the random control, different target melody information can be generated multiple times under the same conditions, so that the target melody information presents diversity.

[0031] In one embodiment, the encoding process of the melody information to be processed and the chord information to be matched according to the preconfigured music theory strategy to obtain the encoding result includes: extracting a first feature vector from the melody information to be processed and the chord information to be matched based on the preconfigured music theory strategy to perform the encoding process to obtain the encoding result. Alternatively, based on the preconfigured music theory strategy, a melody feature close to the target melody information is obtained, and the melody feature is used as a second feature vector to perform the encoding process to obtain the encoding result.

[0032] Among them, the preconfigured music theory strategy can be a simple music theory rule, and based on the music theory rule, it can be possible to analyze from the melody information to be processed and the chord information to be matched: features that may be helpful for predicting the target melody information, encode the features, construct them into a first feature vector (the first feature vector is included in the above encoding result) and use them as the input of the melody generation model; or, by inferring the melody sound that may generate the target melody information as a melody feature based on the preconfigured music theory strategy, the encoding of the melody feature is constructed into a second feature vector (the second feature vector is included in the above encoding result) and use it as the input of the melody generation model.

[0033] In one embodiment, the melody information to be processed and the chord information to be matched are encoded according to a preconfigured music theory strategy, including at least one encoding method selected from basic melody encoding, melody review encoding, chord encoding, position encoding, chord-melody relationship encoding, and type encoding.

[0034] In one embodiment, it also includes: mapping the melody information to be processed into multi-dimensional information to perform the encoding processing of the one-hot encoding; splitting the melody information to be processed into note names according to the tone relationship of the same note name in different octaves to perform the encoding processing of the note name splitting encoding.

[0035] In one embodiment, the method further includes: when the melody information to be processed is repeated at a fixed interval, obtaining a review distance described at the fixed interval; judging whether the first chord information corresponding to all review distance positions is the same as the second chord corresponding to the current position; if they are the same, recording the first chord information that is the same as the second chord as the chord melody relationship code.

[0036] In one embodiment, it also includes: when the melody information to be processed has a melody introduction, development, transition and combination, obtaining the first melody information of the current position in the melody information to be processed, the first melody information is within the position interval formed by the melody introduction, development, transition and combination; and performing the encoding processing of the position encoding according to the relative position of the first melody information within the position interval.

[0037] In one embodiment, the method further includes: obtaining music score information; extracting music segment construction information including phrase information from the music score information according to a preset beat; obtaining a music segment structure according to the music segment construction information, wherein the music segment structure includes a plurality of melody sequences obtained by dividing the melody information to be processed into units of bars. For example, when the bar is the first bar, the position of the first bar is determined according to the first chord at the beginning of the music segment structure. Afterwards, data is collected in units of the music segment structure to obtain a sample data set for training the melody generation model.

[0038] Among them, the sample data set includes: the total number of bars corresponding to the melody information to be processed, the speed corresponding to the melody information to be processed, the classification characteristics of the music segment structure, the position difference of the first note of the melody in the melody information to be processed relative to the first bar, the overall chord of the music segment structure, and the melody of any length at the starting position of the music segment structure.

[0039] In one embodiment, the target melody information corresponding to the chord information to be matched is obtained based on the encoding result and the melody generation model, including: when the melody generation model includes a long short-term memory layer and a fully connected layer, the encoding result is input into the long short-term memory layer to obtain an updated encoding result in which a hidden state is updated in the long short-term memory layer; the updated encoding result is input into the fully connected layer for mapping to obtain a classification probability corresponding to the updated encoding result, and the target melody information is obtained according to the classification probability.

[0040] Application examples:

[0041] The processing flow of the first embodiment of the present application includes the following contents:

[0042] 1. Data Collection, Representation, Preprocessing and Neural Network Model

[0043] Music scores are collected and recorded in segments to form a data set. A segment needs to record the melody and chord progression after transposing to C major or a minor, the first bar position of the melody, the total number of bars b, the speed qpm, and the classification feature t of the segment. The first bar position is determined by the first chord at the beginning of the segment. The first beat of the first bar is defined as the 0 moment of the segment. The position difference of the first note of the melody relative to the first bar is recorded as Δst. If the melody of the segment starts before the first bar, the melody of the advance part is called a weak melody, Δst<0; if it starts on or after the first beat of the first bar, there is no weak melody, Δst≥0.

[0044] Determine the quantization length SPQ of each beat to be 4. For example, for a 4 / 4 song, the melody of each measure is represented by 16 values. Considering the case of a weak start, the melody of a segment needs to be represented by N values, N = 16 × b + Δst, and only the segments that meet -32 ≤ Δst < 16, b ≥ 4 are selected. The melody of a segment is represented by M = m0…m N-1 , where m i ,i∈{0,1,...,N-1} describes the moment If there is a note or rest whose start time is not equal to a certain t i , then adjust its start time to be equal to a nearby t i The entire melody can be shifted octaves so that the melody satisfies:

[0045]

[0046]

[0047] There are N types of melody values. m =62 categories.

[0048] Similarly, the chords that may appear in C major or A minor can be divided into N c = 62 types, the chord progression of a musical passage is expressed as C = c0…c N-1 ,c i ∈{0,1,...,N c -1},i∈{0,1,...,N-1}. i Indicates t i The chord type number currently being played.

[0049] The specific task to be solved is: given the total number of bars b, the tempo qpm, the classification feature t of the music segment, the position difference Δst of the first note of the melody relative to the first bar, and the chord progression c0…c N-1 , a melody of any length starting from the beginning of the segment m0…m k, 0 ≤ k < N, predict the melody m of the remaining part of the music segment k+1 …m N-1 。

[0050] II. Encoding

[0051] Based on simple music theory rules, analyze the features that may be helpful for prediction from the melody and chords, or give possible melody notes as features through reasoning, construct them into a feature vector, and use this feature vector as the input of the neural network model. Specifically, it includes the following multiple melody encoding methods.

[0052] 1) Basic melody encoding

[0053] Encode a melody value a. It is divided into one-hot encoding and pitch name splitting encoding, and one-hot encoding and pitch name splitting encoding can be used. Let One_hot k represent mapping the numerical value to a k-dimensional one-hot vector, and it can ensure that the possible value types of the numerical value do not exceed k. 0 k represents a k-dimensional zero vector.

[0054] One-hot encoding:

[0055] Pitch name splitting encoding: To emphasize the relationship between notes with the same pitch name in different octaves, the encoding is divided into pitch name encoding x pitch , octave encoding x octave , status encoding x status Three parts, then the specific encoding representation is:

[0056]

[0057]

[0058]

[0059] x melody_seperate (a) = concat(x status (a), x octave (a), x status (a))

[0060] 2) Melody lookback encoding

[0061] In pop songs, the technique of repeating the melody at a fixed interval often appears. The repetition interval is usually one and a half beats, two beats, one measure, two measures. In 4 / 4 time, the lookback distance d lookback ∈ D lookback = {6, 8, 16, 32, 64, 128}. When predicting the melody at position i, the one located d before it lookbackThe basic melody encoding of the melody value is used as the melody lookback encoding for the current position. It can more directly input the melody value to be repeated into the model at the current position, thus realizing the repetition technique of the melody. Melody lookback encoding x melody_lookback The calculation method is:

[0062]

[0063] x melody_lookback (i) = concat(x melody_lookback (i, 6), x melody_lookback (i, 8), x melody_lookback (i, 16), x melody_lookb x melody_lookback (i, 64), x melody_lookback (i, 128))

[0064] 3) Chord encoding

[0065] The chord encoding uses one-hot encoding. The possible chords in C major or a minor are divided into N c = 62 categories. The encoding for the chord value a is

[0066]

[0067] 4) Position encoding

[0068] In the melody of a pop song segment, there is a relationship of initiation, continuation, transition, and conclusion, which can generally be attributed to the influence of the relative position of the current melody note within the segment. Encoding this relative position enables the model to generate a melody with a certain structure.

[0069] Denote the k-bit binary encoding of a as BiEncoded(a, k), where 0 ≤ a < 2 k , and its i-th (0 ≤ i < k) dimensional element

[0070] The maximum number of bars in the segment in the data is denoted as b max . When predicting the i-th position m i of the melody sequence, its absolute position p = i + Δst.

[0071] Forward position encoding, as a way of processing position encoding, encodes the binary encoding of the distance of the predicted position relative to the 0 moment of the segment:

[0072]

[0073]

[0074] Reverse position encoding, as another way of processing position encoding, encodes the distance of the predicted position relative to the end of the last bar of the segment. Given the total number of bars b in the segment, we have:

[0075]

[0076] Position function coding, as another processing method of position coding, encodes the possible function of the melody in the whole music segment determined by the current absolute position p. The following features are encoded respectively:

[0077] x neg Indicates whether it is in a weak start measure. An example is as follows:

[0078]

[0079] x part1 Indicates whether it is at the beginning of a segment. An example is as follows:

[0080]

[0081] x part2 Indicates whether the music is in the half-final section. An example is as follows:

[0082]

[0083] x part3 Indicates whether it is in the music segment repetition part, which usually repeats the beginning of the music segment and makes some changes. An example is as follows:

[0084]

[0085] x part4 Indicates whether it is at the end of the segment. An example is as follows:

[0086]

[0087] An example of position function coding is as follows: func (p)=

[0088] (x neg (p),x part1 (p),x part2 (p),x part3 (p),x part4 (p))

[0089] 5) Chord-melody relationship encoding

[0090] It is necessary to combine the encoding of melody and chords.

[0091] The same coding of the chord in the lookback position is used as a processing method for coding the chord-melody relationship. Whether the melody is repeated is closely related to whether the chord begins to repeat. For all lookback distances d lookback ∈

[0092] D lookback The chord at the position is determined to determine whether it is the same as the current chord, and this information is encoded into

[0093] x chord_equal (i) An example is as follows:

[0094] x chord_equal (i)=(x chord_equal (i,6),x chord_equal (i,8),x chord_equal (i,16),x chord_equal (i,32),x chord_equal (i,64),x chord_equal (i,128))

[0095]

[0096] Modulation inference coding, as another way to encode the chord melody relationship, the melody modulation is usually accompanied by a similar modulation relationship with the chord. Only the case where the two melodic notes of the modulation are both natural major and minor tunings and the chord root tone is also natural major and minor tuning is considered. In other cases, the code is a 0 vector. Two notes p1, p2∈{0,1,...,6} in natural major and minor tuning. The numbers correspond to CDEFGAB in C major, and the series difference between the two notes Δp=min(|(p1-p2)mod 7|,|(p2-p1)mod 7|). Modulation distance d sequence ∈D sequence ={8,16,32}. Consider the distance from the predicted position, referred to as the melody of the modular position With chords Predicted position of chord C i With melody m i , if the difference in the degree of the root notes of the two chords is Δp, only the case of Δp∈{1,2} is considered. The difference in the degree of the two melody notes is also Δp. At this time, there are two cases. The predicted melody note is higher than the modulo melody note, which is upward modulo. The predicted melody note is lower than the modulo note, which is downward modulo. scale_alter(a,Δp) means to increase the note a by Δp degrees in C major, using the basic melody code x melody The two inferred modulated sounds are encoded and then merged as modulated inference encoding. An example is as follows:

[0097]

[0098] 6) Type coding

[0099] Speed ​​coding, as a processing method of type coding, the speed range of popular songs is 40≤qpm<220. Large changes in speed have a strong impact on melody and rhythm, but small changes in speed within a certain range do not have a big impact. The speed is classified into 9 different speed types with an interval of 20, and one-hot encoding is performed:

[0100]

[0101] Paragraph type coding, as another way of processing type coding, each music segment has a paragraph type t, and its value and meaning are:

[0102]

[0103] The segment type code is:

[0104] x type (t) = One_hot7(t)

[0105] 3. Training the Neural Network Model

[0106] 1) Data construction for training neural network models

[0107] For each music segment data D = {b, qpm, Δst, t, M, C}, N training data can be constructed, where the input of the k∈{0, 1, ..., N-1}th data is b, qpm, Δst, t, C, m0, ..., m k-1 , the output is m k Encode the inputs as described above and concatenate the results into the model input x at time k k :

[0108] x k =concat(x melody_seperate (m k-1 ),x melody_lookback (k),x chord (c k ),x forward_pos (k+Δst),x forward_pos (k+Δst),x func (k+Δst),x chord_equal (k),x sequence_infer (k),x qpm (qpm),x type (t))

[0109] The output of the model at k

[0110] 2) Structure of the neural network model

[0111] The melody generation model consists of a long short-term memory unit LSTM with a hidden layer of d=256 dimensions and a d×N m The fully connected layer W of dimension is composed of:

[0112] h k =LSTM(h k-1 ,x k )

[0113] y k =Softmax(Wh k )

[0114] At each step, the hidden state h of LSTM at time k-1 is k-1 and k when the input x k Update the hidden state h k , and then the hidden state is mapped to m through the fully connected layer k The distribution of y k .

[0115] During training, adjust the model parameters θ and optimize the following loss function:

[0116]

[0117] in, represents a data set, D represents the data of a music segment, and M represents the melody in D.

[0118] When generated, through y k Sample out m k , and then bring it to the next step for iterative generation.

[0119] This application example extracts features and codes from melodies and chords based on simple music theory rules, gives possible melody sounds as melody input to the neural network model through reasoning, and uses codes instead of complex neural networks to automatically generate melodies. This realizes automatic melody generation using a method that combines music theory rules and deep learning. This ensures that there is no need to write a large number of rules to ensure the diversity and quality of the generated melodies, and that the generated melodies have certain music theory rules, structures, and composition techniques.

[0120] According to an embodiment of the present application, an information processing device is provided. Figure 2 is a schematic diagram of the structure of an information processing device according to an embodiment of the present application. Figure 2As shown, it includes: an encoding module 41, which is used to encode the melody information to be processed and the chord information to be matched according to a preconfigured music theory strategy to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed; a melody generation module 42, which is used to obtain target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model.

[0121] In one embodiment, the encoding module is used to extract a first feature vector from the melody information to be processed and the chord information to be matched based on the preconfigured music theory strategy to perform the encoding process and obtain the encoding result; or, based on the preconfigured music theory strategy, obtain a melody feature close to the target melody information, and use the melody feature as a second feature vector to perform the encoding process and obtain the encoding result.

[0122] In one embodiment, the encoding module is used for at least one encoding method selected from basic melody encoding, melody review encoding, chord encoding, position encoding, chord-melody relationship encoding, and type encoding.

[0123] In one embodiment, a preprocessing module is also included, which is used to map the melody information to be processed into multi-dimensional information to perform the encoding processing of the one-hot encoding; according to the tone relationship of the same note name in different octaves, the melody information to be processed is split into note names to perform the encoding processing of the note name split encoding.

[0124] In one embodiment, the encoding module is further used to obtain the review distance described by the fixed interval when the melody information to be processed is repeated at the fixed interval; determine whether the first chord information corresponding to all review distance positions is the same as the second chord corresponding to the current position; if they are the same, record the first chord information that is the same as the second chord as the chord melody relationship code.

[0125] In one embodiment, the encoding module is also used to obtain first melody information at a current position in the melody information to be processed when the melody information to be processed has a melody introduction, development, transition and combination, and the first melody information is within a position interval formed by the melody introduction, development, transition and combination; and perform encoding processing of the position encoding according to the relative position of the first melody information within the position interval.

[0126] In one embodiment, a training module is further included, which is used to obtain music score information; extract music segment construction information containing phrase information from the music score information according to a preset beat; obtain a music segment structure according to the music segment construction information, wherein the music segment structure includes multiple melody sequences obtained by dividing the melody information to be processed in units of bars; collect data in units of the music segment structure to obtain a sample data set for training the melody generation model; the sample data set includes: the total number of bars corresponding to the melody information to be processed, the speed corresponding to the melody information to be processed, the classification characteristics of the music segment structure, the position difference of the first note of the melody in the melody information to be processed relative to the first bar, the overall chord of the music segment structure, and a melody of any length at the starting position of the music segment structure.

[0127] In one embodiment, the melody generation module is used to input the encoding result into the long short-term memory layer when the melody generation model includes a long short-term memory layer and a fully connected layer, to obtain an updated encoding result for implementing hidden state update in the long short-term memory layer; input the updated encoding result into the fully connected layer for mapping, to obtain a classification probability corresponding to the updated encoding result, and obtain the target melody information according to the classification probability.

[0128] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and will not be repeated here.

[0129] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0130] like Figure 3 , is a block diagram of an electronic device for implementing the information processing method of an embodiment of the present application. The electronic device can be the aforementioned deployment device or proxy device. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples, and are not intended to limit the implementation of the present application described and / or required herein.

[0131] like Figure 3As shown, the electronic device includes: one or more processors 801, memory 802, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. Various components are interconnected using different buses and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the electronic device, including instructions stored in or on the memory to display the graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple electronic devices can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 3 A processor 801 is taken as an example.

[0132] The memory 802 is a non-transient computer-readable storage medium provided in the present application. The memory stores instructions executable by at least one processor to enable the at least one processor to perform the information processing method provided in the present application. The non-transient computer-readable storage medium of the present application stores computer instructions, which are used to enable a computer to perform the information processing method provided in the present application.

[0133] The memory 802 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as program instructions / modules corresponding to the information processing method in the embodiment of the present application. The processor 801 executes various functional applications and data processing of the server by running the non-transient software programs, instructions and modules stored in the memory 802, that is, implementing the information processing method in the above method embodiment.

[0134] The memory 802 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 802 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 802 may optionally include a memory remotely arranged relative to the processor 801, and these remote memories may be connected to the electronic device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0135] The electronic device of the information processing method may further include: an input device 803 and an output device 804. The processor 801, the memory 802, the input device 803 and the output device 804 may be connected via a bus or other means. Figure 3 The example of connecting through bus is taken in the following.

[0136] The input device 803 can receive input digital or character information, and generate key signal input related to user settings and function control of the electronic device, such as a touch screen, a keypad, a mouse, a track pad, a touch pad, an indicator rod, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 804 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0137] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0138] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or means (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0139] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0140] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0141] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0142] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this application can be executed in parallel, sequentially or in different orders, as long as the expected results of the technical solution disclosed in this application can be achieved, and this document is not limited here.

[0143] The above specific implementations do not constitute a limitation on the protection scope of this application. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of this application should be included in the protection scope of this application.

Claims

1. An information processing method, It is characterized in that The method comprises: According to a preconfigured music theory strategy, the melody information to be processed and the chord information to be matched are encoded to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed; Obtaining target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model; The encoding process of the melody information to be processed and the chord information to be matched according to the preconfigured music theory strategy to obtain the encoding result includes: extracting a first feature vector from the melody information to be processed and the chord information to be matched based on the preconfigured music theory strategy to perform the encoding process to obtain the encoding result; or, based on the preconfigured music theory strategy, obtaining a melody feature close to the target melody information, using the melody feature as a second feature vector to perform the encoding process to obtain the encoding result; According to the preconfigured music theory strategy, the melody information to be processed and the chord information to be matched are encoded, including at least one encoding method of basic melody encoding, melody review encoding, chord encoding, position encoding, chord melody relationship encoding, and type encoding.

2. The method according to claim 1, It is characterized in that The encoding process includes one-hot encoding and note name splitting encoding; the method also includes: Mapping the melody information to be processed into multi-dimensional information to perform the encoding process of the one-hot encoding; According to the note relationship of the same note name in different octaves, the to-be-processed melody information is split into note names to perform the encoding process of the note name split encoding.

3. The method according to claim 1, It is characterized in that Also includes: When the melody information to be processed is repeated at a fixed interval, obtaining a review distance described at the fixed interval; It is determined whether the first chord information corresponding to all the review distance positions is the same as the second chord corresponding to the current position. If they are the same, the first chord information that is the same as the second chord is recorded as the chord melody relationship code.

4. The method according to claim 1, It is characterized in that Also includes: When the melody information to be processed has a melody introduction, development, transition and conclusion, obtaining first melody information at a current position in the melody information to be processed, wherein the first melody information is within a position interval formed by the melody introduction, development, transition and conclusion; The encoding process of the position encoding is performed according to the relative position of the first melody information in the position interval.

5. The method according to any one of claims 1 to 4, It is characterized in that Also includes: Get music score information; Extracting music segment construction information including music phrase information from the music score information according to a preset beat; Obtaining a music section structure according to the music section construction information, wherein the music section structure includes a plurality of melody sequences obtained by dividing the melody information to be processed into units of bars; Collecting data based on the music segment structure to obtain a sample data set for training the melody generation model; The sample data set includes: the total number of bars corresponding to the melody information to be processed, the speed corresponding to the melody information to be processed, the classification characteristics of the music segment structure, the position difference of the first note of the melody in the melody information to be processed relative to the first bar, the overall chord of the music segment structure, and a melody of any length at the starting position of the music segment structure.

6. The method according to any one of claims 1 to 4, It is characterized in that The step of obtaining target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model includes: When the melody generation model includes a long short-term memory layer and a fully connected layer, the encoding result is input into the long short-term memory layer to obtain an updated encoding result in which a hidden state is updated in the long short-term memory layer; The updated encoding result is input into the fully connected layer for mapping to obtain a classification probability corresponding to the updated encoding result, and the target melody information is obtained according to the classification probability.

7. An information processing device, It is characterized in that The device comprises: The encoding module is used to encode the melody information to be processed and the chord information to be matched according to a preconfigured music theory strategy to obtain an encoding result; wherein the melody information to be processed includes the total melody length of the melody information to be processed and the beginning part of the melody of the melody information to be processed; A melody generation module, used for obtaining target melody information corresponding to the chord information to be matched according to the encoding result and the melody generation model; The encoding module is further used to: extract a first feature vector from the melody information to be processed and the chord information to be matched based on the preconfigured music theory strategy to perform the encoding process to obtain the encoding result; or, based on the preconfigured music theory strategy, obtain a melody feature close to the target melody information, use the melody feature as a second feature vector to perform the encoding process to obtain the encoding result; The encoding module is also used for at least one encoding method selected from the group consisting of basic melody encoding, melody review encoding, chord encoding, position encoding, chord-melody relationship encoding, and type encoding.

8. The device according to claim 7, It is characterized in that The encoding process includes one-hot encoding and note name splitting encoding; the device also includes a preprocessing module for: Mapping the melody information to be processed into multi-dimensional information to perform the encoding process of the one-hot encoding; According to the note relationship of the same note name in different octaves, the to-be-processed melody information is split into note names to perform the encoding process of the note name split encoding.

9. The device according to claim 7, It is characterized in that The encoding module is further used for: When the melody information to be processed is repeated at a fixed interval, obtaining a review distance described at the fixed interval; It is determined whether the first chord information corresponding to all the review distance positions is the same as the second chord corresponding to the current position. If they are the same, the first chord information that is the same as the second chord is recorded as the chord melody relationship code.

10. The device according to claim 7, It is characterized in that The encoding module is further used for: When the melody information to be processed has a melody introduction, development, transition and conclusion, obtaining first melody information at a current position in the melody information to be processed, wherein the first melody information is within a position interval formed by the melody introduction, development, transition and conclusion; The encoding process of the position encoding is performed according to the relative position of the first melody information in the position interval.

11. The device according to any one of claims 7 to 10, It is characterized in that Also includes training modules for: Get music score information; Extracting music segment construction information including music phrase information from the music score information according to a preset beat; Obtaining a music section structure according to the music section construction information, wherein the music section structure includes a plurality of melody sequences obtained by dividing the melody information to be processed into units of bars; Collecting data based on the music segment structure to obtain a sample data set for training the melody generation model; The sample data set includes: the total number of bars corresponding to the melody information to be processed, the speed corresponding to the melody information to be processed, the classification characteristics of the music segment structure, the position difference of the first note of the melody in the melody information to be processed relative to the first bar, the overall chord of the music segment structure, and a melody of any length at the starting position of the music segment structure.

12. The device according to any one of claims 7 to 10, It is characterized in that The melody generation module is used to: When the melody generation model includes a long short-term memory layer and a fully connected layer, the encoding result is input into the long short-term memory layer to obtain an updated encoding result in which a hidden state is updated in the long short-term memory layer; The updated encoding result is input into the fully connected layer for mapping to obtain a classification probability corresponding to the updated encoding result, and the target melody information is obtained according to the classification probability.

13. An electronic device, include: at least one processor; as well as A memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor so that the at least one processor can execute the method of any one of claims 1-6.

14. A non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method of any one of claims 1 to 6.