Music audio generation method, computer equipment and computer storage medium

Abstract

The embodiment of the invention discloses a music audio generation method, computer equipment and a computer storage medium. The method comprises the steps of generating chord progress and a music structure according to a currently set music scene, determining motivation melody segments from the melody segments according to music features represented by the melody segments, taking the motivation melody segments as music motives corresponding to the music scene, arranging the motivation melody segments according to the music structure, and obtaining motivation motives corresponding to the music scene. And adding at least one interpolated melody fragment into the melody fragment sequence based on the chord to perform motivation development for the music motivation to obtain a music audio file matched with the music scene, thereby performing motivation reproduction for the music motivation to obtain a melody fragment sequence, and adding at least one interpolated melody fragment into the melody fragment sequence based on the chord to perform motivation development for the music motivation to obtain the music audio file matched with the music scene. The arrangement of the music can be flexibly changed according to the listening scene and demand of the user, the user only needs to set the music scene, does not need to change the music arrangement template and does not need to know the terminology of the music arrangement, the time for generating the music is reduced, and the efficiency of generating the music is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of audio processing, and specifically to a music audio generation method, a computer device, and a computer storage medium. Background Art

[0002] In the automatic music generation scheme provided by the related technology, multiple sets of arrangement templates are usually preset by a rule method. A chord progression can be input to generate a complete melody that is harmonious with the chord progression, and the timbre or instrument used in the accompaniment part is specified. However, these schemes have two defects. First, the arrangement templates are fixed and may not meet the actual music needs of users. If the generated music is to meet the user's listening requirements, it takes extra time to change the arrangement template and then use the changed arrangement template to generate the music, which leads to a delay in the generation of the music. Secondly, most of these arrangement templates are described in industry professional terms, and non-professional users cannot understand the meaning behind them. It takes a certain amount of time to learn the meaning of professional terms to understand the function of the arrangement template, which also leads to a delay in the generation of music and low efficiency.

[0003] Therefore, in addition to the time required to use arrangement templates to generate music, the automatic music generation solution of the related art also requires some users to spend more time changing arrangement templates or learning the functions of arrangement templates, which causes delayed music generation and affects the efficiency of music generation. Summary of the invention

[0004] The embodiments of the present application provide a music audio generation method, a computer device, and a computer storage medium for intelligently generating music according to the user's listening scene and needs to meet the user's personalized customization of music.

[0005] A first aspect of an embodiment of the present application provides a method for generating music audio, the method comprising:

[0006] Generate chord progressions and music structures based on the currently set music scene;

[0007] Generate a plurality of melody segments, and determine a motive melody segment from each of the melody segments according to the music features represented by each of the melody segments, wherein the motive melody segment serves as a music motive corresponding to the music scene;

[0008] Arranging the motive melody fragments according to the music structure to reproduce the music motive and obtain a melody fragment sequence;

[0009] At least one interpolated melody segment is added to the melody segment sequence based on the chord progression to develop the music motive and obtain a music audio file matching the music scene.

[0010] A second aspect of an embodiment of the present application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0011] A third aspect of an embodiment of the present application provides a computer storage medium, in which instructions are stored. When the instructions are executed on a computer, the computer executes the method of the first aspect.

[0012] A fourth aspect of the embodiments of the present application provides a computer program product. When the computer program product is run on a computer device, the computer device executes the method of the first aspect.

[0013] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0014] The computer device generates chord progressions and music structures according to the currently set music scene, generates multiple melody fragments, determines the motive melody fragments from each melody fragment according to the music features represented by each melody fragment, and uses the motive melody fragments as the music motives corresponding to the music scene. The motive melody fragments are arranged according to the music structure to reproduce the music motives and obtain a melody fragment sequence. At least one interpolated melody fragment is added to the melody fragment sequence based on the chord progression to develop the music motives and obtain a music audio file that matches the music scene. Therefore, the arrangement of the music can be flexibly changed according to the user's listening scene and needs to meet the user's personalized customization of music. Therefore, the user only needs to set the music scene, without changing the music arrangement template, and does not need to understand the professional terms and music theory knowledge of the music arrangement. The operation process of generating music audio is simplified, the time spent on music generation is reduced, the efficiency of music generation is improved, and the user's listening experience is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic diagram of the network framework in the embodiment of the present application;

[0016] Figure 2 A schematic diagram of a flow chart of a music audio generation method in an embodiment of the present application;

[0017] Figure 3 A flowchart of an optional method for generating an interpolated melody segment corresponding to an interpolated chord in an embodiment of the present application;

[0018] Figure 4 This is an exemplary schematic diagram of a melody fragment in an embodiment of the present application being used as a compositional motive and being repeated and reproduced based on a musical structure;

[0019] Figure 5A schematic diagram of an exemplary display style of an audio density setting interface;

[0020] Figure 6 A schematic diagram of a scene of filling a target audio event in a timeline where a melody segment sequence is located in an embodiment of the present application;

[0021] Figure 7 for Figure 4 An exemplary schematic diagram of each interpolated chord and its corresponding interpolated melody fragment in the example shown;

[0022] Figure 8 An exemplary schematic diagram of feature information describing an audio event corresponding to a note of a musical score in an embodiment of the present application;

[0023] Fig. 9 Schematic diagram of pitch distribution characteristics of two exemplary melody segments in the embodiments of the present application;

[0024] Fig.10 It is a structural schematic diagram of a computer device in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The embodiments of the present application provide a music audio generation method, a computer device, and a computer storage medium for intelligently generating music according to the user's listening scene and needs to meet the user's personalized customization of music.

[0026] See also Figure 1 In the embodiment of the present application, the network framework includes:

[0027] The service server 100 and the terminal cluster; the terminal cluster may include: terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n and other terminal devices.

[0028] The business server 100 may be an independent physical server, a server cluster or a distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud database, cloud service, cloud computing, cloud function, cloud storage, network service, cloud communication, middleware service, domain name service, security service, CDN, and big data and artificial intelligence platform. The terminal device (including terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n) may be a smart phone, tablet computer, laptop computer, desktop computer, PDA, mobile internet device (MID), wearable device (such as smart watch, smart bracelet, etc.), smart computer, smart car and other smart terminals.

[0029] Among them, the business server 100 can establish a communication connection with each terminal device in the terminal cluster, and a communication connection can also be established between each terminal device in the terminal cluster. In other words, the business server 100 can establish a communication connection with each terminal device in the terminal device 200a, terminal device 200b, terminal device 200c, ..., terminal device 200n. For example, a communication connection can be established between the terminal device 200a and the business server 100. A communication connection can be established between the terminal device 200a and the terminal device 200b, and a communication connection can also be established between the terminal device 200a and the terminal device 200c. Among them, the above-mentioned communication connection does not limit the connection method, and can be directly or indirectly connected through a wired communication method, or can be directly or indirectly connected through a wireless communication method, etc., which can be determined according to the actual application scenario, and this application is not limited here.

[0030] It should be understood that Figure 1 Each terminal device in the terminal cluster shown can be installed with an application client. When the application client runs in each terminal device, it can interact with the business server 100 respectively, so that the business server 100 can receive business data from each terminal device (such as user identity data uploaded by the user through the terminal device). Among them, the application client can be a music playback application, a karaoke software application, a browser application, a social application, an instant messaging application, a live broadcast application, a game application, a short video application, a video application, a shopping application, a novel application, a payment application, etc., which has the function of displaying text, images, audio, and video and other data information. The specific application can be determined according to the actual application scenario requirements, and no limitation is made here. Among them, the application client can be an independent client or an embedded sub-client integrated in a client (such as a music application, a karaoke software application, etc.). The specific application can be determined according to the actual application scenario, and no limitation is made here.

[0031] The following will be combined Figure 1 The network framework and the application scenario of the embodiment of the present application are used to describe the music audio generation method in the embodiment of the present application:

[0032] See also Figure 2 In one embodiment of the present application, a method for generating music audio includes:

[0033] 201. Generate chord progression and music structure according to the currently set music scene;

[0034] The method of this embodiment can be applied to a computer device, which can be Figure 1The business server 100 or each terminal device in the network framework shown. In some embodiments, the computer device can implement the music audio generation method provided in this embodiment by running a computer program. For example, the computer program can be a native program or software module in the operating system; it can also be a local (Native) application (APP, Application), that is, a program that needs to be installed in the operating system to run; it can also be a small program, that is, a program that only needs to be downloaded to the browser environment to run; it can also be a small program that can be embedded in any APP. In short, the above-mentioned computer program can be an application, module or plug-in in any form.

[0035] In an optional manner, the computer device may provide a display interface for the user to select a music scene, for example, the scene may include study, sleep, meditation, work, exercise, and other scenes for the user to choose from.

[0036] A chord progression (or "chord direction") refers to the order in which chords are played in a song / music. Different chord progressions have different emotional colors, and chord progressions with different emotional colors can be generated based on the music scene selected by the user. For example, chord progressions suitable for various music scenes or listening scenes can be preset, so that the computer device can determine the chord progression suitable for the music scene selected by the user. Alternatively, a neural network model can be trained by training a large number of audio sets with labeled chromaticity and chord colors to obtain a model for generating chord progressions. The music scene selected by the user is input into the model, and the model outputs the corresponding chord progression. Alternatively, according to the traditional machine learning Markov chain, a Markov matrix is ​​listed to generate chord progressions according to common chord tendencies. This embodiment does not limit the method for generating chord progressions.

[0037] Similarly, the neural network model can be trained by using labeled music structure samples and their corresponding music scenes, so that the model can output the corresponding music structure according to the music scene input by the user; music structures adapted to various music scenes or listening scenes can also be preset, such as pre-setting an ABAB music structure adapted to sleep scenes or meditation scenes, or pre-setting an AAAB music structure adapted to learning scenes or concentration scenes, and so on, so as to determine the music structure adapted to the music scene selected by the user.

[0038] Among them, the ABAB music structure and the AAAB music structure refer to a piece of music consisting of two different melodic sections (section A and section B) that alternate, that is, the order of alternating melodies in a piece of music with an ABAB music structure is ABAB, and the order of alternating melodies in a piece of music with an AAAB music structure is AAAB. This music structure of alternating melodic sections is common in narrative songs, which tell a story through two different themes or melodies.

[0039] 202. Generate a plurality of melody segments, and determine a motive melody segment from each of the melody segments according to the music features represented by each of the melody segments, wherein the motive melody segment serves as a music motive corresponding to the music scene;

[0040] When generating melody fragments, a neural network model such as a recurrent neural network (RNN) or a long short-term memory network (LSTM) can be used to generate a melody fragment of a specific length using MIDI (musical instrument digital interface, a digital interface for musical instruments) as a carrier, which contains musical features such as the pitch, duration, and start time of the notes. All output melody fragments have the same length in the time domain. These melody fragments can be parsed and saved as multiple audio events in the form of a structure in the generation framework. An audio event is used to represent the characteristics of the pitch of a melody fragment. Each audio event corresponds to a pitch, for example, including the pitch name, duration, start time, timbre, end time, and other pitch features of the corresponding pitch.

[0041] Among them, pitch refers to the various tones of notes in a piece of music, that is, the height of the sound, which is one of the basic characteristics of the sound. The pitch name indicates the height of the sound. The pitch name can be expressed by the note names C, D, E, F, G, A, B, or the solfeggios 1, 2, 3, 4, 5, 6, 7.

[0042] After obtaining multiple melody fragments, a motive melody fragment can be determined from the multiple melody fragments according to the music features represented by each melody fragment. For example, a motive melody fragment can be determined based on multiple audio events of each melody fragment. The motive melody fragment serves as a music motive corresponding to a pre-set music scene.

[0043] 203. Arrange the motive melody fragments according to the music structure to reproduce the music motive and obtain a melody fragment sequence;

[0044] In music generation, the recurrence of motives helps maintain the coherence of the work. By repeatedly appearing the same or similar motives, a stable musical image can be established in the minds of the audience, enhancing the recognition of the music. Therefore, in this embodiment, after obtaining the motive melody fragment, the motive can be reproduced based on the motive melody fragment, that is, the motive melody fragments are arranged according to the determined music structure to obtain a melody fragment sequence, thereby realizing the motive reproduction for the music motive.

[0045] 204. Add at least one interpolated melody segment to the melody segment sequence based on the chord progression to develop the music motive and obtain a music audio file matching the music scene;

[0046] After obtaining the melody fragment sequence and its multiple interpolated melody fragments, the melody fragment sequence and its multiple interpolated melody fragments can be output as a music audio file to play the corresponding music. For example, the melody fragment sequence and its multiple interpolated melody fragments can be rendered into an audio file using a synthesizer or related sampling tone files for output and playback, thereby playing music suitable for the scene selected by the user and meeting the user's personalized listening needs.

[0047] After obtaining the melody fragment sequence, at least one interpolated melody fragment can be added to the melody fragment sequence based on the determined chord progression. This method corresponds to repeated notes in the development of the motive of the composition method, thereby realizing the development of the music motive. The melody fragment sequence with the interpolated melody fragment added can be output as an audio file, wherein the interpolated melody fragment refers to a new melody fragment added between adjacent melody fragments. Since the interpolated melody fragment and the melody fragment sequence are generated according to the music scene set by the user, the audio file output according to the interpolated melody fragment and the melody fragment sequence can match the music scene set by the user, meeting the user's personalized listening needs.

[0048] In this embodiment, the computer device generates chord progressions and music structures according to the currently set music scene, generates multiple melody fragments, determines the motive melody fragments from each melody fragment according to the music features represented by each melody fragment, and uses the motive melody fragments as the music motives corresponding to the music scene. The motive melody fragments are arranged according to the music structure to reproduce the music motives and obtain a melody fragment sequence. At least one interpolated melody fragment is added to the melody fragment sequence based on the chord progression to develop the music motives and obtain a music audio file that matches the music scene. Therefore, the arrangement of the music can be flexibly changed according to the user's listening scene and needs to meet the user's personalized customization of music. Therefore, the user only needs to set the music scene, without changing the music arrangement template, and does not need to understand the professional terms and music theory knowledge of the music arrangement, which simplifies the operation process of generating music audio, reduces the time spent on music generation, improves the efficiency of music generation, and thus improves the user's listening experience.

[0049] based on Figure 2In the illustrated embodiment, in a preferred implementation manner, when adding at least one interpolated melody segment to a melody segment sequence, at least one interpolated chord can be added to the melody segment sequence based on a determined chord progression, and an interpolated melody segment corresponding to the interpolated chord is generated, and the melody segment sequence and its multiple interpolated melody segments are output as a music audio file that matches the music scene set by the user.

[0050] Among them, in the melody segment sequence, in order to make the transition between the melody segments more harmonious and smooth, with better connectivity, to achieve a natural transition from one melody segment to another, and to produce a melody gradient effect, for each pair of adjacent melody segments in the melody segment sequence, the interpolation chords corresponding to the adjacent melody segments can be determined in the above-determined chord progression, and the interpolation chords can be chords that are musically consistent with the chords corresponding to the adjacent melody segments, so as to achieve a comfortable listening experience for the melody of the music. For example, according to the arrangement relationship between the main chord and the dominant chord, a chord that is consistent with the chord corresponding to the adjacent melody segment is selected in the chord progression as an interpolation chord. For example, if the chord corresponding to the adjacent melody segment is a C chord, a G chord can be selected in the chord progression to be arranged with it, so that the melody of the music can bring people a comfortable listening experience.

[0051] After the interpolation chord is determined, an interpolation melody segment between two melody segments of the adjacent melody segments may be generated according to the plurality of audio events of each of the adjacent melody segments and the interpolation chord.

[0052] When generating an interpolated melody segment between each pair of adjacent melody segments, for each interpolated chord corresponding to the adjacent melody segments, an interpolated melody segment corresponding to the interpolated chord may be generated. Figure 3 , generating the interpolation melody fragment corresponding to the interpolation chord, including the following steps:

[0053] 301. Determine a plurality of target audio events to be filled corresponding to the interpolated chord according to the plurality of audio events of the melody segment corresponding to the interpolated chord and the pitch of the interpolated chord;

[0054] In this embodiment, the interpolated chord includes at least one interpolated chord corresponding to each melody segment in the adjacent melody segments; the multiple interpolated chords are arranged according to their order in the chord progression; the multiple pitches corresponding to the multiple audio events are arranged in sequence to form a pitch array. It can be determined that the audio event with the same pitch as the pitch of the interpolated chord among the multiple audio events of the melody segment corresponding to the interpolated chord is the target audio event. For each audio event with a different pitch from the pitch of the interpolated chord among the multiple audio events of the melody segment corresponding to the interpolated chord, it is determined whether the pitch of the audio event is replaced with the pitch of the interpolated chord. If so, the pitch of the audio event is replaced with the pitch of the interpolated chord, and the audio event whose pitch has been replaced with the pitch of the interpolated chord is determined as the target audio event; if not, the audio event is determined as the target audio event.

[0055] For example, in Figure 4 In the scene shown, both Motive 1 and Motive 2 correspond to the C chord. Multiple chords can be selected as interpolation chords between Motive 1 and Motive 2 in the chord progression, such as Figure 4 As shown, multiple chords G, Am, Em, F, and G are selected as interpolation chords. Among them, G, Am, and Em chords are assigned as interpolation chords of motive 1, and F and G chords are assigned as interpolation chords of motive 2. In addition, if the number of interpolation chords n is an even number, the first n / 2 chords are interpolation chords of motive 1, and the last n / 2 chords are interpolation chords of motive 2.

[0056] First, read the audio event sequence of motive 1, and filter its pitch information through the interpolated chord assigned to motive 1, that is, determine the audio event with the same pitch as the pitch of its interpolated chord among the multiple audio events of motive 1 as the target audio event. For example, the pitch array corresponding to the multiple audio events of motive 1 is [1, 2, 3], and the pitch contained in the G chord is [5, 7, 2, 4], and the intersection of the two is pitch 2. Therefore, determine the audio event with a pitch of 2 among the multiple audio events of motive 1 as the target audio event.

[0057] Secondly, for each audio event in the multiple audio events of Motive 1 whose pitch is different from the pitch of the G chord, determine whether the pitch of the audio event is replaced with the pitch of the G chord. If so, replace the pitch of the audio event with the pitch of the G chord, and determine the audio event whose pitch has been replaced with the pitch of the G chord as the target audio event; if not, determine the audio event as the target audio event. For example, the pitch of the audio event with a pitch of 1 in Motive 1 may be replaced with any one of the pitches 5, 7, and 4 in the G chord, or it may be retained; the pitch of the audio event with a pitch of 3 in Motive 1 may be replaced with any one of the pitches 5, 7, and 4 in the G chord, or it may be retained.

[0058] Among them, whether the pitch of an audio event in a melody segment is replaced with the pitch of its corresponding interpolated chord can be determined by determining a replacement probability range corresponding to the audio event and generating a random probability. If the random probability is within the replacement probability range corresponding to the audio event, the pitch of the audio event needs to be replaced with the pitch of the interpolated chord; if the random probability exceeds the replacement probability range corresponding to the audio event, the pitch of the audio event is retained, that is, the pitch of the audio event does not need to be replaced.

[0059] By analogy, multiple target audio events can be obtained after motive 1 is filtered and screened by Am chord and Em chord respectively, and multiple target audio events can be obtained after motive 2 is filtered and screened by F chord and G chord respectively.

[0060] 302. Obtain the audio density corresponding to each time position between the adjacent melody segments in the time axis where the melody segment sequence is located;

[0061] In this embodiment, the audio density can be a value within a preset value range, or a value set by the user, which serves as a benchmark for measuring whether the target audio event is filled at the corresponding time position in the time axis. When the audio density is set by the user, the computer device can provide an interface for the user to set the audio density on the interface, receive the user's setting operation on the audio density, and obtain the audio density corresponding to each target audio event according to the setting operation.

[0062] For example Figure 5 As shown, in the interface, the user can adjust the audio density in three levels, and the computer device determines the corresponding audio density according to the user's operation, wherein each level corresponds to a different audio density range, such as a high level corresponds to a higher audio density value range, a medium level corresponds to a medium audio density value range, and a low level corresponds to a lower audio density value range. Of course, the audio density input by the user can also be obtained in other ways, such as the user inputting the audio density value in the input box, or the user selecting the audio density of interest from multiple audio density options, and so on.

[0063] 303. For each of the target audio events, determine a time position for filling the target audio event according to the audio density of each of the time positions in the timeline, and fill the target audio event at the time position for filling the target audio event;

[0064] In this embodiment, the audio density represents the distribution density of multiple target audio events on the time axis, so the greater the audio density, the denser the distribution of multiple target audio events; the smaller the audio density, the more dispersed the distribution of multiple target audio events. For each target audio event, the time position for filling the target audio event can be determined according to the audio density of each time position in the time axis, and the target audio event can be filled in the time position for filling the target audio event.

[0065] When all target audio events are filled at their respective corresponding time positions, all target audio events that have been filled at the corresponding time positions may constitute an interpolated melody segment corresponding to the interpolated chord.

[0066] In a preferred embodiment, in the timeline, the end time of the previous melody segment in the adjacent melody segment is taken as the starting point, and the step length of each step is sequentially advanced with a preset time length, and each time a step is advanced, it is determined whether the random number generated based on the current time position is greater than the audio density corresponding to the current time position. If so, the current target audio event to be filled is filled at the current time position, until all target audio events of the interpolated chord are filled at the corresponding time positions in the timeline, thereby obtaining the interpolated melody segment corresponding to the interpolated chord.

[0067] For example, after determining the multiple target audio events corresponding to the current interpolation chord and the audio density corresponding to each target audio event, it can be determined whether to fill the target audio event at the time position according to the random number corresponding to the time position of each step and the audio density corresponding to the time position. The random number is a value calculated according to a preset algorithm for the current time position. For example, the random number can be determined according to the distance between the current time position and the two melody segments before and after the adjacent melody segment. For example, the functional relationship between the random number and the distance can be a linear functional relationship, such as a linear functional relationship similar to the random number y=ax+bx'+c, where a and b are slopes, c is a constant, x is the distance between the current time position and the previous melody segment of the adjacent melody segment, and x' is the distance between the current time position and the next melody segment of the adjacent melody segment.

[0068] Of course, the functional relationship between the random number and the distance is not limited to the above functional relationship. As long as the functional relationship can indicate that the random number changes with the distance, no limitation is made here.

[0069] Specifically, if the audio density is expressed as a probability value, the random number is specifically a random probability value. Of course, the audio density and the random number may also be other types of values, which are not limited here.

[0070] For example, in Figure 4In the example shown, after motive 1 is filtered by the G chord, it is assumed that the target audio events are the three audio events corresponding to pitches 5, 2 and 7, that is, these three audio events are the target audio events to be filled. The end time of motive 1 can be used as the starting point, and the step length of each step can be 16th notes. The pointer is used to indicate the time position reached in each step, such as Figure 6 As shown, when a step reaches the corresponding time position, it is determined that the random number corresponding to the current time position is greater than the audio density corresponding to the current time position, and an audio event with a pitch of 5 is filled; then, the process continues, and when a step reaches the corresponding time position, it is determined that the random number corresponding to the current time position is greater than the audio density corresponding to the current time position, and an audio event with a pitch of 2 is filled; and the process continues, and similarly, an audio event with a pitch of 7 can be filled at a certain time position. At this point, all target audio events are filled to the corresponding time positions, and all target audio events constitute the interpolated melody fragment corresponding to the G chord between motive 1 and motive 2.

[0071] Similarly, the interpolated melody fragments corresponding to the Am chord, Em chord, F chord, and G chord can be processed, such as Figure 7 The processing shown obtains melody fragments of multiple chords, such as an interpolated melody fragment corresponding to the G chord, an interpolated melody fragment corresponding to the Am chord, and an interpolated melody fragment corresponding to the Em chord. Among them, the all-0 array refers to an array used to pre-occupy part of the storage space in the storage space, and its function is to prevent the storage space from being completely occupied by other data, resulting in no available storage space for the newly generated interpolated melody fragment. Therefore, after the interpolated melody fragment is generated, it is only necessary to replace the all-0 array with the interpolated melody fragment to complete the storage of the interpolated melody fragment.

[0072] Among them, if the audio density is a value within a preset range of values, then in the timeline, the closer the current time position is to one of the motives, the smaller the corresponding audio density value; the farther away from the two motives, the larger the corresponding audio density value. For example, the value range of the audio density can be set to multiple probability value ranges such as [0%~50%].

[0073] Therefore, by adding interpolated melody fragments to the melody fragment sequence based on the audio density, the repeated notes and motive reproduction in the compositional motive development can be achieved, so that the generated melody can ensure structural consistency, the generated melody has a structure that is discernible to the human ear, and the generated melody is more memorable and pleasant to the ear.

[0074] Based on the above embodiments and their preferred embodiments, in another optional embodiment, after all target audio events are filled in the corresponding time positions in the time axis, the offset of each target audio event in the time position of the time axis forward or backward can be obtained, as well as the offset probability of each target audio event; for each target audio event that has been filled in the corresponding time position in the time axis, determine whether to move the time position of the target audio event on the time axis based on the offset of the target audio event according to the offset probability of the target audio event; if so, move the time position of the target audio event on the time axis based on the offset of the target audio event; if not, maintain the time position of the target audio event on the time axis.

[0075] For example, the offset may be one or more 16th notes or other note durations. Assuming that the offset represents a forward shift of 2 16th notes, if it is determined that the target audio event needs to be moved, based on the offset, the target audio event may be shifted forward by 2 16th notes, thereby determining the new time position of the target audio event in the timeline.

[0076] When judging whether to shift the target audio event based on the offset probability, the time position of the target audio event on the time axis can be calculated according to a preset algorithm to obtain a random probability value, and then it is determined whether the random probability value is greater than the offset probability corresponding to the target audio event. If so, it is determined that the target audio event needs to be moved; if not, the time position of the target audio event on the time axis is maintained.

[0077] The random probability value may be determined according to the distance between the current time position of the target audio event and the time position of the adjacent audio event. For example, the functional relationship between the random probability value and the distance may be a linear functional relationship.

[0078] In addition, both the offset amount and the offset probability can be opened for users to customize the settings, so that users can freely adjust the rhythm style of the melody based on their own listening preferences.

[0079] Therefore, by providing an offset function for the time position of the target audio event, the relative distance between audio events in time position can be changed, thereby achieving a change in rhythm in the development of the compositional motive, and the user can independently set the offset amount and offset probability, for example, adjusting the offset amount or offset probability to make the rhythm of the melody more intensive or the rhythm of the melody more soothing, to meet the user's requirements for melody in different listening scenarios or to meet the user's different listening preferences.

[0080] Based on the above-mentioned embodiments and preferred embodiments thereof, in another optional embodiment, when arranging motive melody fragments according to the musical structure to reproduce the musical motive, the similarity between multiple melody fragments can be determined based on the characteristics of the pitch represented by the audio event, and the motive melody fragment is determined in the multiple melody fragments based on the similarity, and a copy of the motive melody fragment is generated, and the motive melody fragment and its copy are arranged according to the determined musical structure to obtain a melody fragment sequence.

[0081] The computer device may determine the similarity between the two melody segments according to the similarity of the pitch characteristics between them, for example, the similarity between the two melody segments may be determined according to the similarity of the pitch name characteristics, the similarity of the pitch start time characteristics, or the similarity of the pitch duration. The method for determining the similarity is not limited, for example, the similarity may be calculated according to the Pearson correlation coefficient formula, or based on the Euclidean distance, Manhattan distance, or other calculation methods.

[0082] Each melody segment is composed of multiple audio events. At this time, the melody segments generated by the network model are filtered and screened according to the chord information generated above and the mode information deduced from the chord information, leaving only the usable melody segments. For example, if the melody segment corresponding to the C chord is generated, the notes in the melody segment that only belong to the C chord will be filtered. After that, the similarity of the filtered melody segments is calculated.

[0083] Specifically, the similarity between two melody segments can be calculated based on any feature of the pitch of the melody segment. In a preferred embodiment, the features such as the pitch name pitch and the start time point start_sample of the audio event can be extracted to calculate the similarity. For every two melody segments, the first similarity between the pitch name feature of one melody segment and the pitch name feature of the other melody segment is determined, and the second similarity between the pitch start time feature of one melody segment and the pitch start time feature of the other melody segment is determined, and the first similarity and the second similarity are weighted and summed, and the weighted sum calculated is used as the similarity of the two melody segments.

[0084] In this embodiment, the audio event represents the pitch characteristics of the melody segment, and each audio event corresponds to a pitch, for example, including the pitch name, duration, start time point, timbre, end time point and other pitch characteristics of the corresponding pitch. Figure 8 As shown, the audio event (note_event) corresponding to a note in a piece of music score can be described as a variety of information such as pitch name pitch, timbre timbre, end mark donesample, duration remainsample and start mark startsample.

[0085] Therefore, based on the above-mentioned various information in the audio event, for example, there is Fig. 9 The two melody segments shown in the figure, where the pitch names of segments 1 and 2 can be replaced by numbers, that is, the pitch names C to B can be replaced according to the following rules: C = 1, D = 2, E = 3, F = 4, G = 5, A = 6, B = 7. In addition, the starting time point is standardized by the length of 16th notes (the length of 16th notes converted from bpm, that is, 16note length = (60 / bpm) / 4), that is, the starting time point is converted into n 16th notes (n can be a floating point number). Therefore, according to the above, the pitch name feature of segment 1 is [1, 2, 3], and the starting time feature of segment 1 is [2, 6, 14]; the pitch name feature of segment 2 is [1, 2, 3, 3], and the starting time feature of segment 2 is [2, 8, 12, 14]. The similarity between two melody segments can be calculated based on these quantized features.

[0086] Of course, the quantization method of the pitch feature of the melody segment is not limited to the above-mentioned example, and the pitch feature can also be quantized into other values ​​based on other quantization methods. In addition, the quantization of the pitch feature is convenient for calculating the similarity between the melody segments. In fact, it is not necessary to quantize the pitch feature, and the similarity between the melody segments can be directly calculated based on the pitch feature, which is not limited here.

[0087] After obtaining the pitch features of the two melody segments, the similarity between the two melody segments can be calculated based on the pitch features. For example, in the above example, the pitch name feature of segment 2 is [1,2,3,3]. Since note repetition is a way of developing musical motives, the pitch array can be deduplicated in its pitch name feature, but the pitch arrangement order cannot be changed. Therefore, the pitch name feature of segment 2 is finally expressed as [1,2,3]. For the pitch start time features with inconsistent lengths, padding operations can be performed according to the longest feature size. For example, the padding operation here can be to randomly select a sampling value in the segment for repetition, but the arrangement order of the pitch corresponding to the start time feature cannot be changed. Therefore, the pitch start time feature of segment 1 can be expanded to [2,2,6,14].

[0088] The Pearson correlation coefficient formula can be used to calculate that the correlation of the pitch name features of segments 1 and 2 is 1, and the correlation of the start time features is 0.87. Different weights can be given to the two features, for example, 1:1, and the final correlation coefficient of segments 1 and 2 is 1*0.5+0.87*0.5=0.935.

[0089] By analogy, the similarities of different groups of melody fragments can be calculated, and one or more groups of melody fragments can be selected according to the similarities as several motive fragments to be repeated in the next music generation.

[0090] Specifically, when determining the motive melody segment from the multiple melody segments according to the similarity, the multiple melody segments with the highest similarity may be determined as the motive melody segment. For example, if every two melody segments form a group, the similarity between the two melody segments in each group may be calculated, and the computer device may determine one or more groups of melody segments with the highest similarity from the multiple groups of melody segments, and generate copies of the one or more groups of melody segments with the highest similarity, and arrange the multiple melody segments with the highest similarity and their copies according to the aforementioned determined music structure to obtain a melody segment sequence.

[0091] For example, assuming that the group of melody fragments with the highest similarity among multiple groups of melody fragments are fragment 1 and fragment 2, and the music structure is determined to be ABAB, then copies of fragment 1 and fragment 2 are generated, and fragment 1, fragment 2 and their copies are arranged according to the above music structure, and the resulting melody fragment sequence is fragment 1-fragment 2-fragment 1-fragment 2-... fragment 1-fragment 2.

[0092] Furthermore, assuming that segment 1 and segment 2 are the two melody segments with the highest similarity, segment 1 can be set as motive 1 and segment 2 can be set as motive 2. Figure 4 In the manner shown, motive 1 and motive 2 are repeated and reproduced according to the predetermined music structure ABAB to obtain the corresponding melody fragment sequence. Of course, fragment 2 can also be set as motive 1, and fragment 1 can be set as motive 2, and repeated and reproduced based on the music structure, which is not limited here.

[0093] Alternatively, multiple melody fragments are grouped together, and the similarity between each pair of melody fragments in each group is calculated. The first n groups of melody fragments (n≥2) with the highest similarity are determined as motive melody fragments, and the motive melody fragments and their copies are arranged to obtain a melody fragment sequence.

[0094] Alternatively, multiple melody segments whose similarity meets a preset similarity threshold are determined as motive melody segments. The preset similarity threshold can be an empirical value obtained by analyzing whether multiple melody segments together constitute a musical motive based on the musical characteristics of the melody segments and summarizing the similarities between multiple groups of melody segments that may constitute a musical motive. It can also be an empirical value summarized based on a large number of experiments, which is not limited here.

[0095] In this embodiment, the arrangement of music can be flexibly changed according to the user's listening scene and needs to meet the user's personalized customization of music. Therefore, the user only needs to set the music scene, without changing the music arrangement template, and without understanding the professional terminology and music theory of music arrangement. The operational process of generating music audio is simplified, the time spent on music generation is reduced, the efficiency of music generation is improved, and the user's listening experience is improved.

[0096] The computer device in the embodiment of the present application is described below. Fig.10 In the embodiment of the present application, one embodiment of the computer device includes:

[0097] The computer device 1000 may include one or more central processing units (CPU) 1001 and a memory 1005 , wherein the memory 1005 stores one or more application programs or data.

[0098] The memory 1005 may be a volatile storage or a persistent storage. The program stored in the memory 1005 may include one or more modules, each of which may include a series of instruction operations in the computer device. Furthermore, the central processing unit 1001 may be configured to communicate with the memory 1005 and execute a series of instruction operations in the memory 1005 on the computer device 1000.

[0099] The computer device 1000 may also include one or more power supplies 1002, one or more wired or wireless network interfaces 1003, one or more input and output interfaces 1004, and / or one or more operating systems, such as Windows ServerTM, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM, etc.

[0100] The CPU 1001 can execute the aforementioned Figures 2 to 3 The operations performed by the computer device in the illustrated embodiment will not be described in detail here.

[0101] The present application also provides a computer storage medium, wherein one embodiment includes: the computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the aforementioned Figures 2 to 3 The operations performed by the computer device in the illustrated embodiment.

[0102] The present application also provides a computer program product, wherein one embodiment includes: when the computer program product is run on a computer device, the computer device executes the aforementioned Figures 2 to 3 The operations performed by the computer device in the illustrated embodiment.

[0103] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0104] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0105] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0106] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0107] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

Claims

1. A method for generating music audio, characterized in that: The method comprises: Generate chord progressions and music structures based on the currently set music scene; Generate a plurality of melody segments, and determine a motive melody segment from each of the melody segments according to the music features represented by each of the melody segments, wherein the motive melody segment serves as a music motive corresponding to the music scene; Arranging the motive melody fragments according to the music structure to reproduce the music motive and obtain a melody fragment sequence; At least one interpolated melody segment is added to the melody segment sequence based on the chord progression to develop the music motive and obtain a music audio file matching the music scene.

2. The method according to claim 1, characterized in that The step of adding at least one interpolated melody segment to the melody segment sequence based on the chord progression to develop the music motive and obtain a music audio file matching the music scene includes: adding at least one interpolated chord to the melody segment sequence based on the chord progression, and generating the interpolated melody segment corresponding to the interpolated chord; The melody segment sequence and the multiple interpolated melody segments thereof are output as a music audio file matching the music scene.

3. The method according to claim 2, characterized in that Each of the melody segments includes a plurality of audio events, and the audio events are used to represent the characteristics of the pitch of the melody segment; The step of adding at least one interpolated chord to the melody segment sequence based on the chord progression and generating the interpolated melody segment corresponding to the interpolated chord comprises: For each pair of adjacent melody segments in the melody segment sequence, determining the interpolated chords corresponding to the adjacent melody segments in the chord progression; The interpolated melody segment between two melody segments of the adjacent melody segments is generated according to the plurality of audio events of each of the adjacent melody segments and the interpolated chord.

4. The method according to claim 3, characterized in that The step of generating the interpolated melody segment between two melody segments of the adjacent melody segments according to the plurality of audio events of each of the adjacent melody segments and the interpolated chords comprises: For each of the interpolated chords corresponding to the adjacent melody segments, determining a plurality of target audio events to be filled corresponding to the interpolated chord according to the plurality of audio events of the melody segment corresponding to the interpolated chord and the pitch of the interpolated chord; Acquire the audio density corresponding to each time position between the adjacent melody segments in the time axis where the melody segment sequence is located; For each of the target audio events, determine a time position for filling the target audio event according to the audio density of each of the time positions in the timeline, and fill the target audio event at the time position for filling the target audio event; Among them, all the target audio events that complete the filling at the corresponding time position constitute the interpolated melody segment corresponding to the interpolated chord.

5. The method according to claim 4, characterized in that The step of determining a time position for filling the target audio event according to the audio density of each time position in the time axis, and filling the target audio event at the time position for filling the target audio event comprises: In the timeline, starting from the end time of the previous melody segment in the adjacent melody segments, proceed sequentially with a preset duration as the step length of each step, and when moving forward each time, determine whether the random number generated based on the current time position is greater than the audio density corresponding to the current time position. If so, fill the target audio event to be filled at the current time position until all the target audio events are filled at the corresponding time positions in the timeline, thereby obtaining the interpolated melody segment corresponding to the interpolated chord.

6. The method according to claim 4, characterized in that The interpolated chords include at least one interpolated chord corresponding to each melody segment in the adjacent melody segments; the plurality of interpolated chords are arranged according to their sequence in the chord progression; The step of determining a plurality of target audio events to be filled corresponding to the interpolated chord according to the plurality of audio events of the melody segment corresponding to the interpolated chord and the pitch of the interpolated chord comprises: Determine, as the target audio event, an audio event having the same pitch as the pitch of the interpolated chord among the multiple audio events of the melody segment corresponding to the interpolated chord; For each audio event of the multiple audio events of the melody fragment corresponding to the interpolated chord, the pitch of the audio event is determined to be replaced by the pitch of the interpolated chord. If so, the pitch of the audio event is replaced by the pitch of the interpolated chord, and the audio event whose pitch has been replaced by the pitch of the interpolated chord is determined as the target audio event; if not, the audio event is determined as the target audio event.

7. The method according to claim 4, characterized in that The obtaining of the audio density corresponding to each time position between the adjacent melody segments in the time axis where the melody segment sequence is located includes: Get the currently set audio density range; For each of the time positions in the time axis, the audio density corresponding to the time position is determined in the audio density range according to the distance between the time position and two preceding and following melody segments in the adjacent melody segments.

8. The method according to claim 4, characterized in that After all the target audio events are filled at corresponding time positions in the timeline, the method further includes: Obtaining an offset of each of the target audio events shifted forward or backward at a time position on the time axis, and obtaining an offset probability of each of the target audio events; For each of the target audio events that have been filled in the corresponding time position on the timeline, determine whether to move the time position of the target audio event on the timeline based on the offset of the target audio event according to the offset probability of the target audio event. If so, move the time position of the target audio event on the timeline based on the offset of the target audio event; otherwise, maintain the time position of the target audio event on the timeline.

9. The method according to claim 8, characterized in that The determining, according to the offset probability of the target audio event, whether to move the time position of the target audio event on the time axis based on the offset of the target audio event includes: Determine a random probability value corresponding to the target audio event according to a distance between a current time position of the target audio event and a time position of an adjacent audio event; If the random probability value is greater than the offset probability of the target audio event, determining to move the time position of the target audio event on the time axis based on the offset of the target audio event; If the random probability value is less than the offset probability of the target audio event, it is determined to maintain the time position of the target audio event on the time axis.

10. The method according to claim 1, characterized in that Each of the melody segments includes a plurality of audio events, and the audio events are used to represent the characteristics of the pitch of the melody segment; The step of arranging the motive melody segments according to the music structure to reproduce the music motive to obtain a melody segment sequence includes: Determining the similarities between two of the plurality of melody segments according to the characteristics of the pitch represented by the audio event; The motive melody segment is determined from the multiple melody segments according to the similarity, and a copy of the motive melody segment is generated. The motive melody segment and its copy are arranged according to the music structure to obtain a melody segment sequence.

11. The method according to claim 10, characterized in that The step of determining the similarities between two of the plurality of melody segments according to the pitch characteristics represented by the audio event comprises: For every two melody segments, a first similarity between the pitch name feature of one melody segment and the pitch name feature of the other melody segment is determined, and a second similarity between the pitch start time feature of one melody segment and the pitch start time feature of the other melody segment is determined, and the first similarity and the second similarity are weightedly summed, and the weighted sum calculated is used as the similarity of the two melody segments.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 11 is implemented.

13. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer is caused to perform the method according to any one of claims 1 to 11.

Images