Music generation method and apparatus, and electronic device and storage medium

By obtaining the initial audio and matching the arrangement template to process the audio, the problems of high cost and poor quality of traditional music generation methods are solved, and the effect of simplifying the production process and improving the quality of music is achieved.

WO2024235301A1PCT designated stage expired Publication Date: 2024-11-21BEIJING ZITIAO NETWORK TECH CO LTD
View PDF 8 Cites 0 Cited by

Patent Information

Application Number
PCT/CN2024/093836
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-05-17
Filing Date
2024-05-17
Publication Date
2024-11-21

AI Technical Summary

Technical Problem

Traditional music generation and production methods have problems of high cost and poor quality, especially in the production process of personal music works, which requires multiple complex production links.

Method used

By obtaining the initial audio and matching the corresponding arrangement template, the audio is processed based on the template to generate a soundtrack with the target music style, thereby simplifying the music production process and improving the music quality.

Benefits of technology

Reduces the difficulty of music production, simplifies the process, improves the quality of the music generated, and makes it easier for individuals to create satisfactory musical works.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN2024093836_21112024_PF_FP_ABST
    Figure CN2024093836_21112024_PF_FP_ABST
Patent Text Reader

Abstract

Provided in the embodiments of the present disclosure are a music generation method and apparatus, and an electronic device, a computer-readable storage medium, a computer program product and a computer program. The method comprises: acquiring an initial audio; acquiring a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used for adding a soundtrack with a target music style to the initial audio; and processing the initial audio on the basis of the first arrangement template, so as to generate target music. After an initial audio recorded by a user is obtained, a first arrangement template, which matches the initial audio, is selected, and the initial audio is processed by using the first arrangement template, thereby adding a soundtrack with a target music style to the initial audio, so as to generate target music, such that the effect of directly processing an initial audio into target music is realized, thus reducing the difficulty of music production, simplifying the production process, and improving the music quality of the generated target music.
Need to check novelty before this filing date? Find Prior Art

Description

Music generation method, device, electronic device and storage medium

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This disclosure claims priority to the Chinese patent application filed with the China Patent Office on May 17, 2023, with application number 202310558414.5 and application name “Music Generation Method, Device, Electronic Device and Storage Medium”, the entire contents of which are incorporated by reference into this disclosure. Technical Field

[0003] The embodiments of the present disclosure relate to the field of Internet technology, and in particular to a music generation method, device, electronic device, computer-readable storage medium, computer program product, and computer program. Background Art

[0004] Currently, creating and sharing personal music is becoming a popular way to spread information. Conventional music works usually require the use of professional equipment and go through multiple production stages such as composing, recording, and arranging to complete a piece of music.

[0005] However, traditional music generation and production methods have the problems of high production costs and poor production quality of music works.

[0006] Summary of the Invention

[0007] Embodiments of the present disclosure provide a music generation method, apparatus, electronic device, computer-readable storage medium, computer program product, and computer program.

[0008] In a first aspect, an embodiment of the present disclosure provides a music generation method, comprising:

[0009] Obtaining initial audio; obtaining a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack with a target musical style to the initial audio; processing the initial audio based on the first arrangement template to generate target music.

[0010] In a second aspect, an embodiment of the present disclosure provides a music generation device, comprising:

[0011] Input module, used to obtain initial audio;

[0012] a processing module, configured to obtain a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack having a target musical style to the initial audio;

[0013] The arrangement module processes the initial audio based on the first arrangement template to generate target music.

[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, including:

[0015] a processor, and a memory communicatively connected to the processor;

[0016] The memory stores computer-executable instructions;

[0017] The processor executes the computer-executable instructions stored in the memory to implement the music generation method described in the first aspect and various possible designs of the first aspect.

[0018] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the music generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0019] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the music generation method described in the first aspect and various possible designs of the first aspect.

[0020] In a sixth aspect, an embodiment of the present disclosure provides a computer program, which, when executed by a processor, implements the music generation method described in the first aspect and various possible designs of the first aspect.

[0021] The music generation method, apparatus, electronic device, computer-readable storage medium, computer program product, and computer program provided in this embodiment obtain initial audio; obtain a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack with a target musical style to the initial audio; and process the initial audio based on the first arrangement template to generate target music. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] FIG1 is a diagram of an application scenario of the music generation method provided by an embodiment of the present disclosure;

[0024] FIG2 is a flowchart of a music generation method according to an embodiment of the present disclosure;

[0025] FIG3 is a flow chart of a specific implementation of step S101 in the embodiment shown in FIG2 ;

[0026] FIG4 is a schematic diagram of a first interface provided by an embodiment of the present disclosure;

[0027] FIG5 is a flowchart of a specific implementation of step S102 in the embodiment shown in FIG2 ;

[0028] FIG6 is a flowchart of a specific implementation method of step S1021 in the embodiment shown in FIG5 ;

[0029] FIG7 is a schematic diagram of another first interface provided by an embodiment of the present disclosure;

[0030] FIG8 is a second flow chart of the music generation method provided by an embodiment of the present disclosure;

[0031] FIG9 is a schematic diagram of displaying pre-generated music according to an embodiment of the present disclosure;

[0032] FIG10 is a schematic diagram of a second interface provided by an embodiment of the present disclosure;

[0033] FIG11 is a flowchart of a specific implementation of step S206 in the embodiment shown in FIG8 ;

[0034] FIG12 is a schematic diagram of jumping from the second interface to the third interface provided by an embodiment of the present disclosure;

[0035] FIG13 is a flowchart of a specific implementation method of step S2062 in the embodiment shown in FIG11 ;

[0036] FIG14 is a flowchart of a specific implementation of step S207 in the embodiment shown in FIG8 ;

[0037] FIG15 is a structural block diagram of a music generating device provided by an embodiment of the present disclosure;

[0038] FIG16 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure;

[0039] FIG17 is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] To make the objectives, technical solutions, and advantages of the embodiments of the present disclosure more clear, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present disclosure without making any creative efforts shall fall within the scope of protection of the present disclosure.

[0041] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0042] The following explains the application scenarios of the embodiments of the present disclosure:

[0043] FIG1 is an application scenario diagram of the music generation method provided by the embodiment of the present disclosure. The music generation method provided by the embodiment of the present disclosure can be applied to the application scenario of generating personal music works. More specifically, as shown in FIG1 , the method provided by the embodiment of the present disclosure can be applied to a terminal device, such as a smart phone, and the music generation method provided by the embodiment is implemented by running a music generation application (Application, APP) on the terminal device. Specifically, for example, after the user triggers the recording component in the music generation application, the terminal device starts to collect the voice emitted by the user, such as a melody or song hummed softly, and then the terminal device automatically generates a soundtrack that matches the user's voice based on the above-collected voice, and after mixing, generates a music work and plays it to enable the user to audition it; and further, if the user is not satisfied with the music work after auditioning, the chords and arrangement of the music work can be further modified until a satisfactory music work is generated.

[0044] Conventional music production typically requires specialized equipment and multiple production steps, including composition, recording, and arrangement. However, traditional music generation and production methods suffer from high production costs and poor quality. The presently disclosed embodiments provide a music generation method to address these issues.

[0045] Referring to FIG2 , FIG2 is a flow chart of a music generation method according to an embodiment of the present disclosure. The method of this embodiment can be applied in a terminal device, and the music generation method includes:

[0046] Step S101: Acquire initial audio.

[0047] For example, in one possible implementation, referring to the application scenario diagram shown in FIG1 , the terminal device first collects sound signals through a sound collection unit, such as a microphone, to obtain the voice emitted by the user, that is, the initial audio. More specifically, for example, the melody or tune hummed by the user, the initial audio may contain only melody information (i.e., only humming the melody), may further contain semantic information (i.e., only reading out the lyrics), or may contain both melody information and semantic information (i.e., humming the melody and lyrics), which is not specifically limited here. By running a target application that can be used to generate a musical work, which is not specifically limited here.

[0048] Furthermore, in one possible implementation, after the terminal device runs the target application, a recording component is provided in the first interface of the target application. In response to a first trigger operation on the recording component, initial audio collection can be achieved. Specifically, as shown in FIG3 , the specific implementation of step S101 includes:

[0049] Step S1011: In response to a first trigger operation within the first interface, real-time voice data is collected.

[0050] Step S1012: After the preset conditions are met, initial audio is generated based on the real-time voice data collected at different times.

[0051] Figure 4 is a schematic diagram of a first interface provided by an embodiment of the present disclosure. The above process is introduced below in conjunction with Figure 4. For example, the first interface is an audio recording interface. The first trigger operation is, for example, clicking or long pressing (shown as a long press operation in the figure) the recording component in the first interface, after which the terminal device begins to continuously collect sound signals and generates real-time voice data based on a time series. The preset conditions are, for example: releasing the long press on the recording component (shown as a release operation in the figure), or after a preset time period. After reaching the preset conditions, the terminal device saves the real-time voice data collected at different times during the above period as input data.

[0052] Optionally, after step S1011, the method further includes:

[0053] Step S1013: Displaying the waveform corresponding to the real-time voice data in real time on the first interface.

[0054] Optionally, after starting to collect real-time voice data, as shown in Figure 4, the waveform corresponding to the current moment is also displayed in the first interface. The waveform is composed of the real-time voice data at the current moment and from the current moment to the collection inspiration moment. Specifically, the waveform is composed of multiple numerical points, each numerical point corresponds to a collection moment, and the value of the numerical point represents the audio amplitude corresponding to the real-time voice data.

[0055] In the steps of this embodiment, the real-time collection of the initial audio is achieved through the first trigger operation on the first interface, which enables flexible control of the recording process of the initial audio and improves the recording efficiency of the initial audio in combination with the waveform display.

[0056] Furthermore, the first interface is further provided with a first setting component, which, before obtaining the initial audio, further includes:

[0057] Step S100: receiving a first setting operation for a first setting component in a first interface, where the first setting operation is used to set a target type of vocal effect.

[0058] Accordingly, in another possible implementation, the specific implementation of step S101 includes:

[0059] Step S1014: sound collection is performed in response to the first trigger operation to obtain original speech.

[0060] Step S1015: Process the original speech to obtain initial audio with the target type of vocal effect.

[0061] Exemplarily, first, the terminal device uses the sound collection unit to collect sound information to obtain the original voice. The specific implementation method can be found in the implementation method of the above steps S1011-S1012, which will not be repeated here. Afterwards, based on the original voice, the original voice is processed, for example, by the vocal effect of the target type selected by the first setting operation, thereby changing the sound characteristics of the original voice. Among them, the vocal effect is the timbre of the human voice when singing, and the type of vocal effect can be represented by the type of music. For example, the type of vocal effect includes pop, jazz, etc. In other possible implementation methods, the type of vocal effect can also be a sub-type further refined based on the above classification, such as city pop, classic pop, etc., which is not specifically limited here.

[0062] Furthermore, the first setting operation may include two sub-operations, which are respectively used to display and trigger different types of vocal effects. For example, when the terminal device responds to the first setting sub-operation input by the user, it triggers the corresponding first setting component and displays several vocal effect identifiers. Thereafter, the terminal device responds to the second setting sub-operation input by the user and selects the corresponding target vocal effect identifier, thereby obtaining the first recording information representing the target type of vocal effect. The first recording information may be a weighted coefficient sequence composed of weighted coefficients for different sound frequencies. The frequency value at at least one frequency point of the original speech is weightedly calculated through the first recording information, thereby obtaining the initial audio with the target type of vocal effect.

[0063] In the steps of this embodiment, by obtaining the first recording information before the initial audio is collected, the initial audio is adjusted so that the initial audio has a personalized vocal effect, thereby improving the listening quality of the target music generated subsequently.

[0064] In another possible implementation, the specific implementation of step S101 includes:

[0065] In response to a first loading operation within the first interface, a voice file is selected; the voice file is loaded to obtain the initial audio. The voice file is a pre-generated file containing a sound signal. After the voice file is obtained locally or remotely through the first loading operation, it is loaded into the terminal device, thereby obtaining a music signal within the voice file, which is equivalent to the original voice obtained in the previous step. Subsequently, the music signal is processed (for example, a vocal effect is set) to obtain the initial audio. The specific implementation method is similar and will not be repeated here.

[0066] Step S102: Acquire a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add music with a target musical style to the initial audio.

[0067] For example, after obtaining the initial audio, a matching arrangement template, i.e., a first arrangement template, is determined based on the special effects of the initial audio, and the soundtrack is added to the initial audio using this first arrangement template. Arrangement refers to the process of configuring elements such as instruments, chords, bass, and harmony based on the main melody. An arrangement template is template data used to implement the above-mentioned music arrangement through a fixed combination. The arrangement template is pre-generated, and based on the characteristics of the initial audio, an arrangement template that matches the characteristics of the initial audio is selected, i.e., the first arrangement template.

[0068] In a possible implementation, as shown in FIG5 , a specific implementation of step S102 includes:

[0069] Step S1021: Acquire first arrangement information of the initial audio, where the first arrangement information represents melody features of the music adapted to the initial audio.

[0070] Step S1022: Obtain a first arrangement template according to the first arrangement information.

[0071] For example, the first arrangement information of the initial audio may be user-defined, automatically generated based on the initial audio, or partially user-defined and partially automatically generated based on the initial audio. In one possible implementation, the first arrangement information represents the melodic characteristics of a soundtrack adapted to the initial audio. The melodic characteristics of the soundtrack may be represented, for example, by the soundtrack's tempo. Therefore, based on the speech tempo of the initial audio, the first arrangement information representing a soundtrack tempo similar to or consistent with the speech tempo can be determined.

[0072] Furthermore, exemplarily, as shown in FIG6 , the specific implementation of step S1021 includes:

[0073] Step S1021A: Obtain the speech rhythm of the initial audio according to the pitch change of the initial audio.

[0074] Step S1021B: Obtain the corresponding music beat according to the speech beat of the initial audio.

[0075] Step S1021C: Obtain first arrangement information according to the music rhythm.

[0076] Exemplarily, the pitch of the initial audio can be obtained by the amplitude of the digital signal corresponding to the initial audio. The method for obtaining the pitch is a prior art and will not be described in detail here. Afterwards, based on the change in pitch between different time points (i.e., pitch change), the speech beat of the initial audio can be obtained. For example, in a certain frequency dimension, the faster the pitch changes, the faster the speech beat. In other words, the slower the pitch changes, the slower the speech beat. Afterwards, based on the speech beat, a similar soundtrack beat is matched, and the corresponding first arrangement information is generated based on the soundtrack beat, wherein the first arrangement information can be a specific beat identifier. At the same time, the preset arrangement template corresponds to a template rhythm identifier that characterizes the speed of its rhythm. By comparing the above beat identifier and the target rhythm identifier, the first arrangement template corresponding to the first arrangement information can be determined.

[0077] Furthermore, in one possible implementation, a second setting component is further provided in the first interface, and the second setting component is used to set the music tempo and / or the playback speed of the initial audio. In another possible implementation, the specific implementation of step S1021 includes:

[0078] Step S1021D: In response to a second setting operation on a second setting component in the first interface, obtaining second recording information, where the second recording information represents a beat of the soundtrack and / or a playback speed of the initial audio;

[0079] Step S1021E: Obtain first arrangement information according to the second recording information.

[0080] After the terminal device responds to the second setting operation input by the user for the second setting component, it can obtain second recording information based on the user's personalized selection; wherein the second recording information represents the beat of the music and / or the playback speed of the initial audio. Figure 7 is a schematic diagram of another first interface provided by an embodiment of the present disclosure. As shown in Figure 7, the second setting component in the first interface includes component A and component B, wherein component A is used to set the beat of the music and component B is used to set the playback speed. When the user clicks on component A and component B respectively, the corresponding setting interface IF_A and setting interface IF_B will pop up. After the user makes specific settings in the setting interface, the terminal device obtains the corresponding beat of the music and the playback speed, thereby obtaining the second recording information. Afterwards, based on the second recording information, the first arrangement information representing the melody characteristics is obtained.

[0081] In the steps of this embodiment, by responding to the second setting operation of the second setting component in the first interface, the personalized setting of the first arrangement information is achieved, thereby meeting the user's personalized arrangement needs, improving the matching degree between the initial audio and the first arrangement template, and improving the music quality of the output target music.

[0082] Step S103: Based on the first arrangement template, process the initial audio to generate target music.

[0083] Exemplarily, after obtaining the first arrangement template based on the above steps, in a possible implementation method, the first arrangement template can directly generate a soundtrack corresponding to the initial audio and having a target music style, wherein, exemplarily, the target music style corresponds to the first arrangement template, and when the first arrangement template is determined based on the first arrangement information, its corresponding target music style is determined. The soundtrack can include multiple soundtrack elements such as instrument sound effects, harmony sound effects, chord melodies, etc. The duration of the soundtrack can be the same as or slightly longer than the duration of the initial audio. After generating the soundtrack corresponding to the initial audio based on the first arrangement template, the soundtrack is mixed with the initial audio to obtain the target music.

[0084] In this embodiment, initial audio is obtained; a first arrangement template corresponding to the initial audio is obtained, wherein the first arrangement template is used to add a soundtrack with a target musical style to the initial audio; and the initial audio is processed based on the first arrangement template to generate the target music. After obtaining the initial audio input by the user, the first arrangement template is selected to match the initial audio, and the first arrangement template is used to process the initial audio, thereby adding a soundtrack with a target musical style to the initial audio, thereby generating the target music. This achieves the effect of directly processing the initial audio into the target music, reducing the difficulty of music production, simplifying the production process, and improving the musical quality of the generated target music.

[0085] Referring to FIG8 , FIG8 is a second flow chart of the music generation method provided by the embodiment of the present disclosure. Based on the embodiment shown in FIG2 , this embodiment further refines step S103 , and the music generation method includes:

[0086] Step S201: collecting sound in response to a first triggering operation to obtain initial audio.

[0087] Step S202: Acquire a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add music with a target musical style to the initial audio.

[0088] Step S203: Obtaining a first arrangement with a target music style according to the first arrangement template.

[0089] For example, referring to the introduction of obtaining the first arrangement template in the embodiment shown in FIG2 , after that, the terminal device, such as the first arrangement template, processes the input music, that is, takes the input music as an input parameter, inputs the first arrangement template, and the first arrangement template uses its ability to generate a soundtrack of the target music style based on the duration of the input music to generate a soundtrack that matches the duration of the input music and has the target music style, that is, the first arrangement. The above-mentioned music style can be expressed through differences in the chords and soundtrack elements that constitute the soundtrack, and the soundtrack elements include instrument sound effects, harmony sound effects, main melody sound effects, etc. That is, the first arrangement has corresponding target chords and / or target soundtrack elements.

[0090] Step S204: Mix the first arrangement and the initial audio to generate pre-generated music.

[0091] Furthermore, after obtaining the first arrangement, the first arrangement and the initial audio are mixed to generate pre-generated music, wherein mixing the first arrangement and the initial audio refers to setting the pitch (intensity of sound energy) of the first arrangement and the initial audio based on a mixing coefficient, and playing the first arrangement and the initial audio simultaneously based on the set pitch, thereby achieving mixing of the first arrangement and the initial audio.

[0092] Among them, the generated pre-generated music is composed of multiple sound channels. Figure 9 is a schematic diagram of displaying pre-generated music provided by an embodiment of the present disclosure. Referring to Figure 9, after the pre-generated music is generated, when the corresponding trigger condition is met (for example, receiving a specific operation instruction from the user), the pre-generated music will be displayed in the display interface. Exemplarily, as shown in Figure 9, the pre-generated music is composed of sound channel chan_1, sound channel chan_2 and sound channel chan_3, among which sound channel chan_1 is a vocal channel, that is, a sound channel carrying the initial audio; sound channel chan_2 and sound channel chan_3 are instrument channels, among which sound channel chan_2 is a sound channel carrying drum sound effects, and sound channel chan_3 is a sound channel carrying keyboard sound effects.

[0093] Optionally, before step S204, the method further includes:

[0094] Step S200A: Displaying a fifth interface, which is used to set a mixing coefficient between the first arrangement and the original audio. The mixing coefficient represents the volume value of each of the first arrangement and the original audio when they are mixed.

[0095] Step S200B: In response to the fifth setting operation on the fifth interface, a target mixing coefficient is obtained.

[0096] For example, the fifth interface may include control components corresponding to the first arrangement and the initial audio, such as a slider component, an editable text box component, etc. After the user inputs a fifth setting operation for one of these control components, the mixing coefficients can be edited, for example, by setting the slider component to a specified position to determine a mixing coefficient. The specific implementation of the fifth setting operation can be configured as needed and is not limited here. The specific manner in which the terminal device obtains the target mixing coefficients in response to the fifth setting operation also varies depending on the fifth setting operation and the aforementioned control components, and will not be further detailed here.

[0097] Accordingly, the specific implementation of step S204 includes:

[0098] The first arrangement and the initial audio are mixed based on the target mixing coefficient to generate pre-generated music. The specific implementation method of mixing based on the mixing coefficient has been introduced above and will not be repeated here. It should be noted that the fifth interface is displayed based on the user's trigger instruction. For example, after the user clicks the trigger component corresponding to the fifth interface, the user jumps from the second interface to the fifth interface. Alternatively, the user jumps to the fifth interface from the third interface, the fourth interface, etc. The above steps S200A and S200B can also be executed again based on the user's operation after executing step S204.

[0099] Optionally, after step S204, the method further includes:

[0100] Step S205A: In the second interface, a first template identifier corresponding to the first arrangement template and a second template identifier corresponding to at least one alternative arrangement template are displayed, wherein the first template identifier and the at least one second template identifier are arranged based on a target arrangement order, and the target arrangement order is determined based on at least the first arrangement information of the initial audio, and the first arrangement information represents the melodic characteristics of the soundtrack adapted to the initial audio.

[0101] Step S205B: In response to the selection operation on the candidate arrangement template, an updated first arrangement is generated.

[0102] In one possible implementation, the pre-generated music is generated by the terminal device after processing the initial audio through the matched first editing target. In this implementation, the user can complete the generation of the pre-generated music and the final target music by only inputting the initial audio (for example, humming a simple melody), simplifying the process to the greatest extent, reducing the complexity and difficulty of user operations, and improving the melody of the arrangement. However, on this basis, the user can further edit the soundtrack of the pre-generated music based on the audition of the automatically generated pre-generated music, thereby achieving a more personalized soundtrack arrangement. Specifically, for example, after generating the pre-generated music, the audio tracks of the pre-generated music can be further edited, such as adding, deleting audio tracks, or separating audio tracks. For another example, the pre-generated music can be edited in a more detailed manner, such as changing the chords, instruments, etc. of the pre-generated music.

[0103] FIG10 is a schematic diagram of a second interface provided by an embodiment of the present disclosure. As shown in FIG10 , pre-generated music is automatically generated on a terminal device. On the one hand, the waveform corresponding to the pre-generated music is displayed in the first area of ​​the second interface, wherein the horizontal axis is the time axis, and the playback position shown in the figure is 00:12, that is, the 12th second of the playback timestamp of the pre-generated music. The vertical axis is the amplitude of the pre-generated music at each time point. On the other hand, the second area of ​​the second interface displays the currently selected arrangement template, i.e., the first arrangement target, and at least one alternative arrangement target. The first template identifier and at least one second template identifier are arranged based on the target arrangement order, and the target arrangement order is determined based on at least the first arrangement information of the initial audio. In short, the arrangement templates are arranged according to the degree of match between each arrangement template and the initial audio. The higher the match, the closer the arrangement position is (closer to the left edge of the second area). The calculation method of the matching degree of each arrangement template can be referred to the description of the implementation method of obtaining the first arrangement target in the embodiment shown in FIG2 .

[0104] Furthermore, when the user clicks on one of the alternative arrangement templates, the terminal device responds to the user's selection operation, determines the selected alternative arrangement target as the first arrangement template, and regenerates an updated first arrangement based on the updated first arrangement template, and returns to step S204, mixing the updated first arrangement with the initial audio to regenerate new pre-generated music.

[0105] Optionally, after step S204, the method further includes:

[0106] Step S206: updating the music elements of the pre-generated music, thereby generating an updated first arrangement.

[0107] For example, in addition to the above-mentioned step of updating the arrangement template, the music elements in the arrangement template can also be updated, which is equivalent to further personalizing the arrangement template, thereby obtaining a first arrangement that better meets the user's needs. For example, the music elements include at least one of the following: instrument sound effects, harmony sound effects, main melody sound effects, and background sound effects. For example, as shown in Figure 11, the specific implementation method of step S206 includes:

[0108] Step S2061: Based on the target playback position of the pre-generated music, a third interface is displayed, where the third interface is used to display the music elements of the first arrangement corresponding to the pre-generated music at the target playback position.

[0109] Step S2062: In response to the third setting operation on the third interface, a second arrangement is obtained.

[0110] Exemplarily, during the process of displaying and playing pre-generated music in the second interface, when the pre-generated music plays the target playback position, or jumps (seek) to the target playback position based on the user operation, the terminal device displays the third interface after receiving the third interface trigger operation input by the user. The third interface is used to display the soundtrack elements of the first arrangement corresponding to the pre-generated music at the target playback position. Afterwards, the user can implement the third setting operation based on the third interface to realize the setting of the soundtrack elements, such as changing the instrument sound effects, changing the harmony sound effects, etc. FIG12 is a schematic diagram of a jump from a second interface to a third interface provided by an embodiment of the present disclosure. As shown in FIG12 , pre-generated music is displayed in the second interface. When the pre-generated music is played to time t_1 (playing timestamp), the user clicks the editing component in the second interface to jump to the third interface, and the third interface displays the music elements within the time period starting from time t_1 and ending at time t_1+det, where det is a value greater than or equal to the minimum adoption interval, representing the editing interval of the music elements. The music elements are implemented through sound channels. For example, as shown in the figure, the music elements include drum sound effects (the corresponding element is identified as Rhy_1), background sound effects (the corresponding element is identified as Amb_1), bass sound effects (the corresponding element is identified as Bas_1), harmony sound effects (the corresponding element is identified as Harm_1), and main melody sound effects (the corresponding element is identified as Lea_1). Each music element corresponds to a sound channel, and the sound channel carries the specific sound data of the specific music element, that is, the implementation method.

[0111] Furthermore, when the user applies a third setting operation to the music element (corresponding component) in the third interface, the terminal device will change the specific implementation of the music element according to the setting instruction corresponding to the third setting operation, thereby generating a new arrangement, namely the second arrangement.

[0112] Exemplarily, the third setting operation includes at least a first sub-operation and a second sub-operation performed sequentially. As shown in FIG13 , a specific implementation of step S2062 includes:

[0113] Step S2062A: In response to the first sub-operation for the target music element, at least two candidate element identifiers corresponding to the target music element are displayed, where the candidate element identifiers represent implementation methods of the target music element.

[0114] Step S2062B: In response to the second sub-operation on the target element identifier among the at least two candidate element identifiers, the target music element is set as the target implementation mode.

[0115] Step S2062C: Obtain a second arrangement based on the target implementation method of the target music element and the corresponding implementation methods of other music elements.

[0116] For example, in conjunction with the third interface shown in FIG12 , when a bass sound effect is selected as the target soundtrack element, a first sub-operation for the element identifier corresponding to the bass sound effect displays multiple candidate element identifiers in the form of a drop-down menu. Then, based on the second sub-operation, one of the candidate element identifiers is selected as the target element identifier (shown as Bas_1 in the figure) and displayed. Simultaneously, the target implementation method corresponding to the target element identifier of the bass sound effect is combined with the current implementation methods of the drum sound effect, harmony sound effect, background sound effect, and main melody sound effect to obtain the second arrangement.

[0117] Step S2063: The second arrangement is determined as the updated first arrangement.

[0118] For example, the second arrangement is determined as the updated first arrangement, and the process can return to step S204 to update the pre-generated music. In this embodiment, by modifying the soundtrack elements in response to the third setting operation on the third interface, the second arrangement is obtained, thereby updating the pre-generated music, further improving the arrangement flexibility and meeting the user's personalized needs.

[0119] Step S207: updating the chords of the pre-generated music, thereby generating an updated first arrangement.

[0120] For example, as shown in FIG14 , the specific implementation of step S207 includes:

[0121] Step S2071: Based on the target playback position of the pre-generated music, a fourth interface is displayed, where the fourth interface is used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position.

[0122] Step S2072: In response to the fourth setting operation on the fourth interface, a third arrangement is obtained.

[0123] Step S2073: Mixing is performed based on the third arrangement and the initial audio to obtain updated pre-generated music.

[0124] For example, similar to the way of setting the soundtrack elements, in the process of displaying and playing the pre-generated music in the second interface, when the pre-generated music plays the target playback position, or jumps to the target playback position based on the user operation (seek), the terminal device displays the fourth interface after receiving the fourth interface trigger operation input by the user. The fourth interface is used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position. After that, the user can implement the fourth setting operation based on the fourth interface to achieve the setting of the chords. Specifically, a chord refers to a group of sounds with a certain interval relationship, that is, three or more notes are combined vertically according to the superposition relationship of thirds or non-thirds, which is called a chord. Chords generally include triads (chords of three notes), seventh chords (chords of four notes), and ninth chords (five notes). Based on the different implementation forms of the chords, a corresponding number of syllable components can be displayed in the fourth interface. Based on the fourth setting operation, the syllable content corresponding to the syllable component is changed to achieve different syllable combinations, thereby achieving different chords. The specific display method and specific setting method of the fourth interface can be set as needed and are specifically limited here.

[0125] The second arrangement is determined as the updated first arrangement, and the process returns to step S204 to update the pre-generated music. In this embodiment, by modifying the chords in response to the fourth setting operation on the fourth interface, a third arrangement is obtained, thereby updating the pre-generated music, further improving the arrangement flexibility and meeting the user's personalized needs.

[0126] Step S208: In response to the second trigger operation, the pre-generated music is exported to generate the target music.

[0127] For example, after the aforementioned steps of generating the pre-generated music at least once are completed, the personalized pre-generated music that meets the user's needs is exported, for example, through the export component in the second interface shown in FIG12 , to generate directly playable target music, such as music files in MP3 or WAV format, thereby completing the process of creating a musical work. Optionally, at the same time, a music cover that matches the target music can be further generated, such as an image or video that matches the soundtrack of the target music, as the music cover, and the target application and the music cover are published. The specific implementation process is not further described here.

[0128] In this embodiment, based on the automatic generation of the first editing template, the automatically selected editing template can be further modified based on the user's setting operations, and the music elements, chords and mixing coefficients can be adjusted to achieve more detailed music settings, thereby generating a music work that meets the user's personalized needs.

[0129] In this embodiment, the implementation of step S201-step S202 is the same as the implementation of step S101-step S102 in the embodiment shown in FIG. 2 of the present disclosure, and will not be described in detail here.

[0130] Corresponding to the music generation method of the above embodiment, FIG15 is a block diagram of the structure of the music generation device provided by the embodiment of the present disclosure. For ease of explanation, only the parts related to the embodiment of the present disclosure are shown. Referring to FIG15, the music generation device 3 includes:

[0131] Input module 31, used to obtain initial audio;

[0132] The processing module 32 is configured to obtain a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack having a target musical style to the initial audio;

[0133] The arrangement module 33 processes the initial audio based on the first arrangement template to generate target music.

[0134] In one embodiment of the present disclosure, the input module 31 is specifically used to: collect real-time voice data in response to a first trigger operation within the first interface; after a preset condition is met, generate initial audio based on the real-time voice data collected at different times; the input module 31 is also used to: display the waveform corresponding to the real-time voice data in real time within the first interface.

[0135] In one embodiment of the present disclosure, before obtaining the initial audio, the input module 31 is also used to: receive a first setting operation for a first setting component in the first interface, the first setting operation being used to set a target type of vocal effect; the input module 31 is specifically used to: collect sound in response to the first trigger operation to obtain the original speech; and process the original speech to obtain initial audio with the target type of vocal effect.

[0136] In one embodiment of the present disclosure, the arrangement module 33 is specifically used to: obtain a first arrangement with a target music style based on a first arrangement template; mix the first arrangement and the initial audio to generate pre-generated music; and export the pre-generated music in response to a second trigger operation to generate target music.

[0137] In one embodiment of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module 33 is further used for at least one of the following: displaying a first template identifier corresponding to the first arrangement template and a second template identifier corresponding to at least one alternative arrangement template in a second interface, wherein the first template identifier and the at least one second template identifier are arranged based on a target arrangement order, and the target arrangement order is determined at least based on the first arrangement information of the initial audio, and the first arrangement information represents the melodic characteristics of the soundtrack adapted to the initial audio; and displaying a waveform corresponding to the pre-generated music in the second interface.

[0138] In one embodiment of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module 33 is further used to: display a third interface based on the target playback position of the pre-generated music, the third interface being used to display the soundtrack elements of the first arrangement corresponding to the pre-generated music at the target playback position; obtain a second arrangement in response to a third setting operation on the third interface; and obtain updated pre-generated music based on the second arrangement and the initial audio.

[0139] In one embodiment of the present disclosure, the third setting operation includes at least a first sub-operation and a second sub-operation that are executed in sequence. When the arrangement module 33 obtains the second arrangement in response to the third setting operation for the third interface, it is specifically used to: in response to the first sub-operation for the target soundtrack element, display at least two alternative element identifiers corresponding to the target soundtrack element, and the alternative element identifiers represent the implementation method of the target soundtrack element; in response to the second sub-operation for the target element identifier among the at least two alternative element identifiers, set the target soundtrack element to the target implementation method; and obtain the second arrangement based on the target implementation method of the target soundtrack element and the implementation methods corresponding to other soundtrack elements.

[0140] In one embodiment of the present disclosure, the music elements include at least one of the following: instrument sound effects, harmony sound effects, and main melody sound effects.

[0141] In one embodiment of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module 33 is further used to: display a fourth interface based on the target playback position of the pre-generated music, the fourth interface being used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position; obtain a third arrangement in response to a fourth setting operation on the fourth interface; and obtain updated pre-generated music based on the third arrangement and the initial audio.

[0142] In one embodiment of the present disclosure, before mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module 33 is further used to: display a fifth interface, the fifth interface is used to set the mixing coefficient of the first arrangement and the initial audio, the mixing coefficient representing the respective volume values ​​of the first arrangement and the initial audio when they are mixed; in response to the fifth setting operation on the fifth interface, obtain the target mixing coefficient; when mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module 33 is specifically used to: mix the first arrangement and the initial audio based on the target mixing coefficient to generate pre-generated music.

[0143] In one embodiment of the present disclosure, the processing module 32 is specifically configured to: obtain first arrangement information of the initial audio, where the first arrangement information represents the melody characteristics of the music adapted to the initial audio; and obtain a first arrangement template based on the first arrangement information.

[0144] In one embodiment of the present disclosure, when the processing module 32 obtains the first arrangement information of the initial audio, it is specifically used to: obtain the voice beat of the initial audio according to the pitch change of the initial audio; obtain the corresponding background music beat according to the voice beat of the initial audio; and obtain the first arrangement information according to the background music beat.

[0145] In one embodiment of the present disclosure, when obtaining the first arrangement information of the initial audio, the processing module 32 is specifically used to: obtain second recording information in response to a second setting operation on a second setting component in the first interface, the second recording information representing the soundtrack beat and / or the playback speed of the initial audio; and obtain the first arrangement information based on the second recording information.

[0146] The input module 31, processing module 32, and arrangement module 33 are connected in sequence. The music generating device 3 provided in this embodiment can implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not be repeated here.

[0147] FIG16 is a schematic structural diagram of an electronic device provided by an embodiment of the present disclosure. As shown in FIG16 , the electronic device 4 includes:

[0148] A processor 41, and a memory 42 communicatively connected to the processor 41;

[0149] Memory 42 stores computer-executable instructions;

[0150] The processor 41 executes the computer-executable instructions stored in the memory 42 to implement the music generation method in the embodiments shown in Figures 2 to 14.

[0151] Optionally, the processor 41 and the memory 42 are connected via a bus 43 .

[0152] The relevant explanations can be understood by referring to the relevant descriptions and effects corresponding to the steps in the embodiments corresponding to Figures 2 to 14, and no further details will be given here.

[0153] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the music generation method provided in any one of the embodiments corresponding to Figures 2 to 14 of the present disclosure.

[0154] The present disclosure provides a computer program product, including a computer program. When the computer program is executed by a processor, the music generation method shown in the embodiments of FIG. 2 to FIG. 14 is implemented.

[0155] Referring to FIG17 , a schematic diagram of the structure of an electronic device 900 suitable for implementing an embodiment of the present disclosure is shown. The electronic device 900 may be a terminal device or a server. The terminal device may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (e.g., vehicle-mounted navigation terminals), and fixed terminals such as digital TVs and desktop computers. The electronic device shown in FIG17 is merely an example and should not limit the functionality and scope of use of the embodiments of the present disclosure.

[0156] As shown in Figure 17, the electronic device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. Various programs and data required for the operation of the electronic device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0157] Typically, the following devices can be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication device 909 can allow the electronic device 900 to communicate with other devices wirelessly or by wire to exchange data. Although FIG17 shows an electronic device 900 with various devices, it should be understood that it is not required to implement or have all of the devices shown. More or fewer devices may be implemented or have alternatively.

[0158] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network via the communication device 909, or installed from the storage device 908, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are performed.

[0159] It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. In the present disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, radio frequencies (RF), etc., or any suitable combination thereof.

[0160] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0161] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0162] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet Service Provider).

[0163] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0164] The units involved in the embodiments described in this disclosure may be implemented in software or hardware. In some cases, the name of a unit does not limit the unit itself. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses."

[0165] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Parts (ASSPs), System on Chips (SOCs), Complex Programmable Logic Devices (CPLDs), and the like.

[0166] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0167] In a first aspect, according to one or more embodiments of the present disclosure, a music generation method is provided, comprising:

[0168] Obtaining initial audio; obtaining a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack with a target musical style to the initial audio; processing the initial audio based on the first arrangement template to generate target music.

[0169] According to one or more embodiments of the present disclosure, obtaining the initial audio includes: collecting real-time voice data in response to a first trigger operation within a first interface; after a preset condition is met, generating the initial audio based on the real-time voice data collected at different times; the method also includes: displaying the waveform corresponding to the real-time voice data in real time within the first interface.

[0170] According to one or more embodiments of the present disclosure, before obtaining the initial audio, it also includes: receiving a first setting operation for a first setting component in a first interface, wherein the first setting operation is used to set a target type of vocal effect; obtaining the initial audio includes: collecting sound in response to a first trigger operation to obtain original speech; processing the original speech to obtain initial audio with the target type of vocal effect.

[0171] According to one or more embodiments of the present disclosure, processing the initial audio based on a first arrangement template to generate target music includes: obtaining a first arrangement having the target music style according to the first arrangement template; mixing the first arrangement and the initial audio to generate pre-generated music; and exporting the pre-generated music in response to a second trigger operation to generate the target music.

[0172] According to one or more embodiments of the present disclosure, after the first arrangement and the initial audio are mixed to generate pre-generated music, at least one of the following items is also included: in a second interface, a first template identifier corresponding to the first arrangement template and a second template identifier corresponding to at least one alternative arrangement template are displayed, wherein the first template identifier and at least one second template identifier are arranged based on a target arrangement order, and the target arrangement order is determined based on at least the first arrangement information of the initial audio, and the first arrangement information represents the melodic characteristics of the soundtrack adapted to the initial audio; in the second interface, a waveform corresponding to the pre-generated music is displayed.

[0173] According to one or more embodiments of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, it also includes: displaying a third interface based on the target playback position of the pre-generated music, the third interface being used to display the soundtrack elements of the first arrangement corresponding to the pre-generated music at the target playback position; obtaining a second arrangement in response to a third setting operation on the third interface; and mixing based on the second arrangement and the initial audio to obtain updated pre-generated music.

[0174] According to one or more embodiments of the present disclosure, the third setting operation includes at least a first sub-operation and a second sub-operation performed in sequence, and the second arrangement is obtained in response to the third setting operation on the third interface, including: in response to the first sub-operation on the target soundtrack element, displaying at least two alternative element identifiers corresponding to the target soundtrack element, and the alternative element identifiers represent the implementation method of the target soundtrack element; in response to the second sub-operation on the target element identifier among the at least two alternative element identifiers, setting the target soundtrack element as the target implementation method; and obtaining the second arrangement based on the target implementation method of the target soundtrack element and the implementation methods corresponding to other soundtrack elements.

[0175] According to one or more embodiments of the present disclosure, the music elements include at least one of the following: instrument sound effects, harmony sound effects, and main melody sound effects.

[0176] According to one or more embodiments of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, it also includes: displaying a fourth interface based on the target playback position of the pre-generated music, the fourth interface being used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position; obtaining a third arrangement in response to a fourth setting operation on the fourth interface; and mixing based on the third arrangement and the initial audio to obtain updated pre-generated music.

[0177] According to one or more embodiments of the present disclosure, before mixing the first arrangement and the initial audio to generate pre-generated music, the method further includes: displaying a fifth interface, the fifth interface being used to set a mixing coefficient of the first arrangement and the initial audio, the mixing coefficient representing the respective volume values ​​of the first arrangement and the initial audio when they are mixed; obtaining a target mixing coefficient in response to a fifth setting operation on the fifth interface; mixing the first arrangement and the initial audio to generate pre-generated music, including: mixing the first arrangement and the initial audio based on the target mixing coefficient to generate pre-generated music.

[0178] According to one or more embodiments of the present disclosure, obtaining a first arrangement template corresponding to the initial audio includes: obtaining first arrangement information of the initial audio, the first arrangement information representing the melodic characteristics of the soundtrack adapted to the initial audio; and obtaining the first arrangement template based on the first arrangement information.

[0179] According to one or more embodiments of the present disclosure, obtaining the first arrangement information of the initial audio includes: obtaining the speech beat of the initial audio based on the pitch change of the initial audio; obtaining the corresponding background music beat based on the speech beat of the initial audio; and obtaining the first arrangement information based on the background music beat.

[0180] According to one or more embodiments of the present disclosure, obtaining the first arrangement information of the initial audio includes: obtaining second recording information in response to a second setting operation on a second setting component in the first interface, the second recording information representing a soundtrack beat and / or a playback speed of the initial audio;

[0181] The first arrangement information is obtained according to the second recording information.

[0182] In a second aspect, according to one or more embodiments of the present disclosure, there is provided a music generating apparatus, comprising:

[0183] Input module, used to obtain initial audio;

[0184] a processing module, configured to obtain a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack having a target musical style to the initial audio;

[0185] The arrangement module processes the initial audio based on the first arrangement template to generate target music.

[0186] According to one or more embodiments of the present disclosure, the input module is specifically used to: collect real-time voice data in response to a first trigger operation within the first interface; after a preset condition is met, generate the initial audio based on the real-time voice data collected at different times; the input module 31 is also used to: display the waveform corresponding to the real-time voice data in real time within the first interface.

[0187] According to one or more embodiments of the present disclosure, before obtaining the initial audio, the input module is further used to: receive a first setting operation for a first setting component within a first interface, wherein the first setting operation is used to set a target type of vocal effect; the input module is specifically used to: collect sound in response to a first trigger operation to obtain original speech; and process the original speech to obtain initial audio with the target type of vocal effect.

[0188] According to one or more embodiments of the present disclosure, the arrangement module is specifically used to: obtain a first arrangement with the target music style based on the first arrangement template; mix the first arrangement and the initial audio to generate pre-generated music; and export the pre-generated music in response to a second trigger operation to generate the target music.

[0189] According to one or more embodiments of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module is further used for at least one of the following: displaying a first template identifier corresponding to the first arrangement template and a second template identifier corresponding to at least one alternative arrangement template in a second interface, wherein the first template identifier and at least one second template identifier are arranged based on a target arrangement order, and the target arrangement order is determined based on at least first arrangement information of the initial audio, the first arrangement information representing the melodic characteristics of the soundtrack adapted to the initial audio; and displaying a waveform corresponding to the pre-generated music in the second interface.

[0190] According to one or more embodiments of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module is further used to: display a third interface based on the target playback position of the pre-generated music, the third interface being used to display the soundtrack elements of the first arrangement corresponding to the pre-generated music at the target playback position; obtain a second arrangement in response to a third setting operation on the third interface; and obtain updated pre-generated music based on the second arrangement and the initial audio.

[0191] According to one or more embodiments of the present disclosure, the third setting operation includes at least a first sub-operation and a second sub-operation that are executed in sequence. When the arrangement module obtains the second arrangement in response to the third setting operation for the third interface, it is specifically used to: in response to the first sub-operation for the target soundtrack element, display at least two alternative element identifiers corresponding to the target soundtrack element, and the alternative element identifiers represent the implementation method of the target soundtrack element; in response to the second sub-operation for the target element identifier among the at least two alternative element identifiers, set the target soundtrack element as the target implementation method; and obtain the second arrangement based on the target implementation method of the target soundtrack element and the implementation methods corresponding to other soundtrack elements.

[0192] According to one or more embodiments of the present disclosure, the music elements include at least one of the following: instrument sound effects, harmony sound effects, and main melody sound effects.

[0193] According to one or more embodiments of the present disclosure, after mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module is further used to: display a fourth interface based on the target playback position of the pre-generated music, the fourth interface being used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position; obtain a third arrangement in response to a fourth setting operation on the fourth interface; and obtain updated pre-generated music based on the third arrangement and the initial audio.

[0194] According to one or more embodiments of the present disclosure, before mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module is further used to: display a fifth interface, wherein the fifth interface is used to set a mixing coefficient of the first arrangement and the initial audio, wherein the mixing coefficient represents the respective volume values ​​of the first arrangement and the initial audio when they are mixed; obtain a target mixing coefficient in response to a fifth setting operation on the fifth interface; and when mixing the first arrangement and the initial audio to generate pre-generated music, the arrangement module is specifically used to: mix the first arrangement and the initial audio based on the target mixing coefficient to generate pre-generated music.

[0195] According to one or more embodiments of the present disclosure, the processing module is specifically configured to: obtain first arrangement information of the initial audio, where the first arrangement information represents melodic features of music adapted to the initial audio; and obtain the first arrangement template based on the first arrangement information.

[0196] According to one or more embodiments of the present disclosure, when the processing module obtains the first arrangement information of the initial audio, it is specifically used to: obtain the speech beat of the initial audio according to the pitch change of the initial audio; obtain the corresponding background music beat according to the speech beat of the initial audio; and obtain the first arrangement information according to the background music beat.

[0197] According to one or more embodiments of the present disclosure, when the processing module obtains the first arrangement information of the initial audio, it is specifically used to: obtain second recording information in response to a second setting operation on a second setting component in the first interface, wherein the second recording information represents the soundtrack beat and / or the playback speed of the initial audio; and obtain the first arrangement information based on the second recording information.

[0198] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0199] The memory stores computer-executable instructions;

[0200] The processor executes the computer-executable instructions stored in the memory to implement the music generation method described in the first aspect and various possible designs of the first aspect.

[0201] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the music generation method described in the first aspect and various possible designs of the first aspect is implemented.

[0202] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the music generation method described in the first aspect and various possible designs of the first aspect.

[0203] In a sixth aspect, an embodiment of the present disclosure provides a computer program, which, when executed by a processor, implements the music generation method described in the first aspect and various possible designs of the first aspect.

[0204] The above description is merely a preferred embodiment of the present disclosure and an illustration of the technical principles employed. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by the specific combination of the above-mentioned technical features, but also includes other technical solutions formed by any combination of the above-mentioned technical features or their equivalents without departing from the above-mentioned disclosed concepts. For example, a technical solution formed by replacing the above-mentioned features with (but not limited to) technical features with similar functions disclosed in this disclosure.

[0205] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details have been included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0206] Although the subject matter has been described in language specific to structural features and / or methodological logical acts, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are merely example forms of implementing the claims.

Claims

1. A music generation method, comprising: Get the initial audio; Acquire a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack having a target music style to the initial audio; Based on the first arrangement template, the initial audio is processed to generate target music.

2. The method according to claim 1, wherein: The obtaining of the initial audio includes: In response to a first trigger operation in the first interface, collecting real-time voice data; After the preset conditions are met, the initial audio is generated based on the real-time voice data collected at different times; The method further comprises: The waveform corresponding to the real-time voice data is displayed in real time in the first interface.

3. The method according to claim 2, wherein: Before collecting real-time voice data in response to a first trigger operation in the first interface, the method further includes: Receiving a first setting operation for a first setting component in a first interface, wherein the first setting operation is used to set a target type of vocal effect; The collecting of real-time voice data in response to a first trigger operation in the first interface includes: In response to the first trigger operation, sound collection is performed to obtain original speech; The original speech is processed to obtain initial audio with the target type of vocal effect.

4. The method according to claim 1, wherein: The obtaining of the initial audio includes: In response to a first loading operation in the first interface, selecting a voice file; The voice file is loaded to obtain the initial audio.

5. The method according to any one of claims 1 to 4, wherein: The step of processing the initial audio based on the first arrangement template to generate target music includes: Obtaining a first arrangement having the target music style according to the first arrangement template; Mixing the first arrangement and the initial audio to generate pre-generated music; In response to a second trigger operation, exporting the pre-generated music to generate the target music; Before generating the target music, the method further includes: The pre-generated music is edited to obtain updated pre-generated music.

6. The method according to claim 5, wherein: After mixing the first arrangement and the initial audio to generate pre-generated music, the method further includes at least one of the following: In the second interface, a first template identifier corresponding to the first arrangement template and a second template identifier corresponding to at least one candidate arrangement template are displayed, wherein the first template identifier and at least one second template identifier are arranged based on a target arrangement order, and the target arrangement order is determined based on at least first arrangement information of the initial audio, and the first arrangement information represents a melody feature of a soundtrack adapted to the initial audio; In the second interface, the waveform corresponding to the pre-generated music is displayed.

7. The method according to claim 5, wherein: After mixing the first arrangement and the initial audio to generate pre-generated music, the method further includes: Based on the target playback position of the pre-generated music, displaying a third interface, the third interface being used to display the soundtrack elements of the first arrangement corresponding to the pre-generated music at the target playback position; In response to a third setting operation on the third interface, a second arrangement is obtained; Mixing is performed based on the second arrangement and the initial audio to obtain updated pre-generated music.

8. The method according to claim 7, wherein: The third setting operation at least includes a first sub-operation and a second sub-operation performed in sequence, and the second arrangement is obtained in response to the third setting operation on the third interface, including: In response to a first sub-operation on a target music score element, at least two candidate element identifiers corresponding to the target music score element are displayed, wherein the candidate element identifiers represent implementation methods of the target music score element; In response to a second sub-operation on a target element identifier among the at least two candidate element identifiers, setting the target music score element as a target implementation mode; The second arrangement is obtained based on the target implementation method of the target music element and the corresponding implementation methods of other music elements.

9. The method according to claim 7 or 8, wherein: The music elements include at least one of the following: Instrumental sound effects, harmony sound effects, main melody sound effects.

10. The method according to claim 5, wherein: After mixing the first arrangement and the initial audio to generate pre-generated music, the method further includes: Based on the target playback position of the pre-generated music, displaying a fourth interface, the fourth interface being used to display the chords of the first arrangement corresponding to the pre-generated music at the target playback position; In response to a fourth setting operation on the fourth interface, a third arrangement is obtained; Mixing is performed based on the third arrangement and the initial audio to obtain updated pre-generated music.

11. The method according to any one of claims 5 to 10, wherein: Before mixing the first arrangement and the initial audio to generate pre-generated music, the method further includes: Displaying a fifth interface, wherein the fifth interface is used to set a mixing coefficient of the first arrangement and the initial audio, wherein the mixing coefficient represents the volume value of each of the first arrangement and the initial audio when they are mixed; In response to a fifth setting operation on the fifth interface, obtaining a target mixing coefficient; Mixing the first arrangement and the initial audio to generate pre-generated music includes: The first arrangement and the initial audio are mixed based on the target mixing coefficient to generate pre-generated music.

12. The method according to any one of claims 1 to 11, wherein: The obtaining of a first arrangement template corresponding to the initial audio includes: Acquire first arrangement information of the initial audio, where the first arrangement information represents melody features of music adapted to the initial audio; The first arrangement template is obtained according to the first arrangement information.

13. The method according to claim 12, wherein: The obtaining of the first arrangement information of the initial audio includes: Obtaining the speech beat of the initial audio according to the pitch change of the initial audio; According to the speech beat of the initial audio, a corresponding music beat is obtained; The first arrangement information is obtained according to the music rhythm.

14. The method according to claim 12, wherein: The obtaining of the first arrangement information of the initial audio includes: In response to a second setting operation on a second setting component in the first interface, second recording information is obtained, The second recording information represents the beat of the soundtrack and / or the playing speed of the initial audio; The first arrangement information is obtained according to the second recording information.

15. A music generating device, comprising: An input module, used to obtain initial audio; A processing module, configured to obtain a first arrangement template corresponding to the initial audio, wherein the first arrangement template is used to add a soundtrack having a target music style to the initial audio; The arrangement module processes the initial audio based on the first arrangement template to generate target music.

16. An electronic device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the music generation method according to any one of claims 1 to 14.

17. A computer-readable storage medium, wherein: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the music generation method according to any one of claims 1 to 14 is implemented.

18. A computer program product, comprising a computer program, which implements the music generating method according to any one of claims 1 to 14 when executed by a processor.

19. A computer program, which, when executed by a processor, implements the music generating method according to any one of claims 1 to 14.

Citation Information

Patent Citations

  • Method for automatically generating music melody on basis of template

    CN104485101A

  • Processing method and device of music file, terminal and storage medium

    CN110211556A

  • Soundtrack method, mobile terminal, data processing method and soundtrack system

    CN110634465A

  • Information processing method and device, electronic equipment and storage medium

    CN114005424A

  • Audio determination method and device, electronic equipment and storage medium

    CN115831080A