Method, device and storage medium for drum sound generation

Abstract

The application provides a drum audio generation method, device and storage medium, and relates to the technical field of artificial intelligence. The method comprises the following steps: detecting an audio time length setting operation of a user; generating a drum performance instruction file corresponding to a drum audio of a target time length based on a drum performance instruction generation model, wherein the target time length is indicated by the audio time length setting operation, and the drum performance instruction file comprises rhythm information of at least one drum type; detecting a drum beat rhythm adjustment operation of the user; adjusting the rhythm information of the at least one drum type based on the drum beat rhythm adjustment operation; and generating drum audio based on the adjusted rhythm information of the at least one drum type. The application can generate personalized drum audio.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of audio technology, in particular to a drum audio generation method, device and storage medium. BACKGROUND

[0002] In music, drums, as the main force of low-frequency instruments, can bring rhythm and rhythm to music. In addition, the high transient burst of drums also makes drums the backbone of music.

[0003] In the music production process, the overall style and genre of the music can be adjusted by editing drum audio. How to generate personalized drum audio for users according to user needs has become a problem to be solved at present. SUMMARY

[0004] The embodiments of the present application provide a drum audio generation method, device and storage medium, which can generate personalized drum audio according to user needs. The technical solution is as follows:

[0005] In a first aspect, a drum audio generation method is provided, which includes:

[0006] Detecting an audio duration setting operation of a user account;

[0007] Generating a drum performance instruction file corresponding to drum audio of a target duration based on a drum performance instruction generation model, wherein the target duration matches the duration indicated by the audio duration setting operation, and the drum performance instruction file includes rhythm information corresponding to each drum type in at least one drum type, wherein the rhythm information corresponding to each drum type is used to indicate the sound condition of the corresponding drum type in each unit time;

[0008] Detecting a drum point rhythm adjustment operation of the user account;

[0009] Adjusting the rhythm information of each drum type based on the drum point rhythm adjustment operation;

[0010] Generating drum audio based on the adjusted rhythm information of each drum type.

[0011] In a possible implementation manner, the adjusting the rhythm information of each drum type based on the drum point rhythm adjustment operation includes:

[0012] Generating an initial encoding matrix based on the drum performance instruction file, wherein an element in the nth row and the mth column of the initial encoding matrix is used to indicate the sound condition of the nth drum type in the mth time unit included in the drum performance instruction file;

[0013] Updating at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation;

[0014] generating drum audio based on the adjusted rhythm information of each drum type, comprises:

[0015] generating drum audio based on the updated encoding matrix.

[0016] In a possible implementation, the updating at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation comprises:

[0017] In a case where the drum point rhythm adjustment operation is a drum point dense adjustment operation, in the initial encoding matrix, a part of non-sound elements are selected according to a controlled probability corresponding to each element, wherein a sound condition indicated by the non-sound element is non-sound, and the controlled probability corresponding to each element is determined based on whether a beat corresponding to each element is a strong beat and whether an element adjacent to each element is a sound element;

[0018] the part of non-sound elements are all modified to sound elements, wherein a sound condition indicated by the sound element is sound;

[0019] In a case where the drum point rhythm adjustment operation is a drum point sparse adjustment operation, in the encoding matrix, a part of sound elements are selected according to a controlled probability corresponding to each element;

[0020] the part of sound elements are all updated to non-sound elements.

[0021] In a possible implementation, the updating at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation comprises:

[0022] In a case where the drum point rhythm adjustment operation is a drum point fast rhythm adjustment operation, a time unit is reduced;

[0023] updating the encoding matrix according to the reduced time unit;

[0024] In a case where the drum point rhythm adjustment operation is a drum point slow rhythm adjustment operation, a time unit is increased;

[0025] updating elements in the encoding matrix according to the increased time unit.

[0026] In a possible implementation, the method further comprises:

[0027] displaying an audio duration setting interface, wherein the audio duration setting interface comprises a custom time option;

[0028] The detecting an audio duration setting operation of a user account comprises:

[0029] detect a selection operation of a user account on the custom time option;

[0030] display a custom time sub-interface;

[0031] detect an audio duration set by the user account in the custom time sub-interface;

[0032] determine the target duration based on the audio duration.

[0033] In a possible implementation, the method further includes:

[0034] display an audio playing interface, wherein the audio playing interface includes a tuning option;

[0035] The detection of the drum beat rhythm adjustment operation of the user account includes:

[0036] detect a selection operation of the user account on the tuning option;

[0037] display a tuning sub-interface, wherein the tuning sub-interface includes a drum rhythm adjustment option;

[0038] detect an adjustment operation of the user account on the drum beat rhythm adjustment option;

[0039] The adjustment of the rhythm information of each drum type based on the drum beat rhythm adjustment operation includes:

[0040] adjust the rhythm information of each drum type based on the adjustment operation on the drum beat rhythm adjustment option.

[0041] In a possible implementation, the drum performance instruction file is a musical instrument digital interface (MIDI) file.

[0042] In a second aspect, a drum audio generation apparatus is provided, and the apparatus includes:

[0043] a detection module configured to detect an audio duration setting operation of a user account;

[0044] a generation module configured to generate, based on a drum performance instruction generation model, a drum performance instruction file corresponding to drum audio of a target duration, wherein the target duration matches a duration indicated by the audio duration setting operation, and the drum performance instruction file includes rhythm information corresponding to each drum type in at least one drum type, and the rhythm information corresponding to each drum type is used to indicate a sound emission condition of the corresponding drum type in each unit time;

[0045] The detection module is further configured to detect a drum beat rhythm adjustment operation of the user account.

[0046] The generation module is further configured to adjust the rhythm information of each drum type based on the drum point rhythm adjustment operation, and generate drum audio based on the adjusted rhythm information of each drum type.

[0047] In a possible implementation, the generation module is configured to:

[0048] generate an initial encoding matrix based on the drum performance indication file, where an element in the nth row and the mth column of the initial encoding matrix is used to indicate a sounding condition of an nth drum type included in the drum performance indication file at an mth time unit;

[0049] update at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation;

[0050] generate drum audio based on the updated encoding matrix of the initial encoding matrix.

[0051] In a possible implementation, the generation module is configured to:

[0052] in a case where the drum point rhythm adjustment operation is a drum point dense adjustment operation, select, in the initial encoding matrix, part of non-sounding elements according to a controlled probability corresponding to each element, where the non-sounding element indicates a non-sounding condition, and the controlled probability corresponding to each element is determined based on whether a beat corresponding to each element is a strong beat and whether an adjacent element of each element is a sounding element;

[0053] modify the part of non-sounding elements to sounding elements, where the sounding element indicates a sounding condition;

[0054] in a case where the drum point rhythm adjustment operation is a drum point sparse adjustment operation, select, in the encoding matrix, part of sounding elements according to a controlled probability corresponding to each element;

[0055] update the part of sounding elements to non-sounding elements.

[0056] In a possible implementation, the generation module is configured to:

[0057] in a case where the drum point rhythm adjustment operation is a drum point fast rhythm adjustment operation, reduce a time unit;

[0058] update the encoding matrix according to the reduced time unit;

[0059] in a case where the drum point rhythm adjustment operation is a drum point slow rhythm adjustment operation, increase a time unit;

[0060] According to the increased time unit, elements in the encoding matrix are updated.

[0061] In a possible implementation, the detection module is further configured to:

[0062] display an audio duration setting interface, wherein the audio duration setting interface includes a custom time option;

[0063] detect a selection operation of the custom time option by the user account;

[0064] display a custom time sub-interface;

[0065] detect an audio duration set by the user account in the custom time sub-interface;

[0066] determine the target duration based on the audio duration.

[0067] In a possible implementation, the detection module is further configured to:

[0068] display an audio playing interface, wherein the audio playing interface includes a tuning option;

[0069] detect a selection operation of the tuning option by the user account;

[0070] display a tuning sub-interface, wherein the tuning sub-interface includes a drum rhythm adjustment option;

[0071] detect an adjustment operation of the drum rhythm adjustment option by the user account;

[0072] The generation module is configured to:

[0073] adjust the rhythm information of each drum type based on the adjustment operation of the drum rhythm adjustment option.

[0074] In a possible implementation, the drum performance indication file is a musical instrument digital interface (MIDI) file.

[0075] In a third aspect, a computing device is provided, which includes a processor and a memory, and the memory stores at least one instruction, which is loaded and executed by the processor to implement the operations performed by the method for generating drum audio according to the first aspect and any possible implementation of the first aspect.

[0076] In a fourth aspect, a computer-readable storage medium is provided, which stores at least one instruction, which is loaded and executed by a processor to implement the operations performed by the method for generating drum audio according to the first aspect and any possible implementation of the first aspect.

[0077] In a fifth aspect, a computer program product is provided, in which at least one instruction is stored, the instruction is loaded and executed by a processor to implement the operation performed by the method for generating drum audio according to the first aspect and any possible implementation of the first aspect.

[0078] The technical scheme provided in the application has the beneficial effects that:

[0079] In the technical scheme provided in the application, the user can set the duration of the drum audio through the audio duration setting operation according to actual needs, and the user can also adjust the drum beat rhythm of at least one drum type included in the drum performance instruction file generated by the drum performance instruction generation model through the drum beat rhythm adjustment operation according to actual needs, and finally generate the drum audio. It can be seen that the drum audio generated by the scheme is personalized according to user needs. BRIEF DESCRIPTION OF DRAWINGS

[0080] In order to more clearly illustrate the technical schemes in the embodiments of the application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the application, and other drawings can be obtained by those skilled in the art without creative labor.

[0081] Figure 1 is a method flowchart for generating drum audio provided by an embodiment of the application;

[0082] Figure 2 is a method flowchart for generating drum audio provided by an embodiment of the application;

[0083] Figure 3 is an interface schematic diagram of a music application program provided by an embodiment of the application;

[0084] Figure 4 is an interface schematic diagram of a music application program provided by an embodiment of the application;

[0085] Figure 5 is an interface schematic diagram of a music application program provided by an embodiment of the application;

[0086] Figure 6 is an interface schematic diagram of a music application program provided by an embodiment of the application;

[0087] Figure 7 is an interface schematic diagram of a music application program provided by an embodiment of the application;

[0088] Figure 8 is a schematic diagram of an element corresponding to a controlled probability provided by an embodiment of the application;

[0089] Figure 9 is a structural schematic diagram of a terminal provided by an embodiment of the present application.

[0090] Figure 10 is a structural schematic diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0091] To make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0092] The embodiment of the present application provides a drum audio generation method, which can be implemented by a computing device, and the computing device can be a terminal or a server, wherein the terminal can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, etc., and the server can be a single server, a server cluster, etc.

[0093] The drum audio generation method provided by the embodiment of the present application is described below, which can be implemented by a computing device, and the computing device can be a terminal or a server, wherein the terminal can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, etc., and the server can be a single server, a server cluster, etc. Figure 1 In the technical solution provided by the embodiment of the present application, the computing device can generate a drum performance instruction file through a neural network model according to an instruction issued by a user, and then encode and convert the drum performance instruction file to a symbol domain (such as a matrix). Then, the symbol domain is edited and updated according to the user instruction, and finally, the symbol domain is decoded to generate drum audio.

[0094] The drum audio generation method provided by the embodiment of the present application is described below, which can be implemented by a computing device, and the computing device can be a terminal or a server, wherein the terminal can be a mobile phone, a tablet computer, a desktop computer, a notebook computer, etc., and the server can be a single server, a server cluster, etc. Figure 2 The processing flow of the method can include the following steps:

[0095] Step 101, detecting an audio time length setting operation of a user account.

[0096] In implementation, a music application program can be installed in the computing device, and a user can log in a user account in the music application program. The music application program can have a music therapy function option, and the user can operate the user account to select the music therapy function option. After the computing device detects the selection operation of the music therapy function option by the user account, the interface of the music application program is switched to a use scenario selection interface. In the use scenario selection interface, at least one use scenario option is included, for example, concentration, sleep, meditation, etc. The user can select one target use scenario option according to actual needs, and after the computing device detects the selection operation of the target use scenario option by the user account, the interface of the music application program is switched to an audio time length setting interface of the target use scenario, wherein the audio time length setting interface can include a custom time option, as shown in Figure 3

[0097] ​The user can operate the user account to select the custom time option. After the computing device detects the selection operation of the user on the custom time option, a custom time sub-interface is displayed, as shown in Figure 4 Further, the user can operate the user account to set the audio duration in the custom time sub-interface. Further, the computing device obtains the audio duration set by the user.

[0098] Step 102, generating a drum performance instruction file corresponding to the drum audio of the target duration based on the drum performance instruction generation model.

[0099] The drum performance instruction generation model is a neural network model, the target duration matches the duration indicated by the audio duration setting operation, and the drum performance instruction file includes rhythm information corresponding to each drum type in at least one drum type. The rhythm information corresponding to each drum type is used to indicate the sound condition of the corresponding drum type in each unit time.

[0100] In implementation, after the computing device obtains the audio duration set by the user, the target duration for generating the drum audio is determined according to the audio duration. For example, the audio duration can be N times of the target duration, and N can be a positive integer, or the audio duration can be the target duration minus a preset duration. The preset duration can be configured by a technician according to actual needs.

[0101] Then, the computing device inputs the target duration into the drum performance instruction generation model, and the drum performance instruction generation model generates a drum performance instruction file corresponding to the drum audio of the target duration. In a possible implementation, the computing device can also input the target duration and the target use scenario identifier into the drum performance instruction generation model, and the drum performance instruction generation model generates a drum performance instruction file corresponding to the drum audio of the target duration and suitable for the target use scenario.

[0102] In a possible implementation, the above drum performance instruction file can be a MIDI (Musical Instrument Digital Interface) file. The rhythm information of each drum type in the drum audio is recorded in the MIDI file, for example, the sound condition of the bass drum every time unit, the sound condition of the snare drum every time unit, and the sound condition of the military drum every time unit. The time units corresponding to different drum types are the same, for example, the time unit is 1s. The sound condition can be whether to sound or not, or the striking intensity. The striking intensity can be divided into multiple levels, for example, it can be divided into three levels, no striking (i.e. no sound), medium intensity, and high intensity.

[0103] Step 103, detecting a drum point rhythm adjustment operation of the user account.

[0104] In implementation, after generating the drum performance instruction file, the computing device can generate initial drum audio based on the file and redirect the music application to an audio playback interface. This interface can include playback options and tuning options, such as... Figure 5 As shown. Users can first select playback options. After detecting the user's selection, the computing device plays the initial drum audio. Users can also decide whether to adjust the drum rhythm according to their needs. If the user wants to adjust the drum rhythm, they can select a tuning option. After detecting the user's selection of the tuning option, the computing device redirects the music application to a tuning sub-interface. The tuning sub-interface includes drum rhythm adjustment options, which users can adjust. The computing device can detect user adjustments to the drum rhythm adjustment options. Specifically, drum rhythm adjustment can include adjusting drum density and drum rhythm speed. Correspondingly, the drum rhythm adjustment options can include drum density adjustment options and drum rhythm speed adjustment options, such as... Figure 5 As shown.

[0105] Users can adjust the drum beat density by sliding the drum beat density adjustment option, such as... Figure 5 As shown, if the user slides the drum density adjustment option to the left, the computing device can detect the drum sparseness adjustment operation; if the user slides the drum density adjustment option to the right, the computing device can detect the drum density adjustment operation.

[0106] Users can adjust the drum tempo by sliding the drum tempo adjustment option, such as... Figure 6 As shown, if the user slides the drum beat speed adjustment option to the left, the computing device can detect the drum beat speed adjustment operation; if the user slides the drum beat speed adjustment option to the right, the computing device can detect the drum beat speed adjustment operation.

[0107] Step 104: Adjust the rhythm information for each drum type based on the drum rhythm adjustment operation.

[0108] In implementation, the computing device can encode the drum performance instruction file to obtain an initial encoding matrix. The element in the nth row and mth column of this initial encoding matrix indicates the rhythmic information of the nth drum type included in the drum performance instruction file at the mth time unit. This encoding converts the drum performance instruction file into a symbol field, facilitating subsequent adjustments to the drum type's rhythmic information based on the user's drumbeat rhythm adjustments.

[0109] The encoding matrix is ​​explained below:

[0110] In a drum performance instruction file where the rhythmic information of the drum shape indicates whether the drum shape has sounded after each time unit, the encoding matrix can be as follows:

[0111]

[0112] Among them, a e1.t1 This indicates whether drum type e1 produces sound at time t1, where t1 is a unit of time, t2 is twice t1, and so on. e1.t1 The value of a is 0 or 1. e1.t1 When the value is 0, it indicates that drum type e1 does not produce sound at time t1. e1.t1 When the value is 1, it indicates that drum type e1 emits sound at time t1.

[0113] When the rhythm information of the drum pattern in the drum performance instruction file is the striking force of the drum pattern over one time unit, the encoding matrix can be as follows:

[0114]

[0115] Where i0, i1, and i2 represent three striking intensities: i0 represents no striking, i1 represents a medium striking, and i2 represents a high striking. i0 is silent, while i1 and i2 both produce sound. e1.t1.i0 This represents drum type e1, which is not struck at time t1. When the striking force is i0, the corresponding element can have a value of 00, such as a. e1.t1.i0 The value can be 0. When the striking force is i1, the corresponding element can have a value of 0 or 1, such as a. e1.t1.i1 The value can be 0 or 1. When the striking force is i2, the corresponding element value can be 10, such as a. e2.t2.i2 The value can be 10.

[0116] When the computing device detects that the drum rhythm adjustment operation is a dense drum rhythm adjustment operation, it selects a portion of silent elements in the initial encoding matrix according to the controlled probability corresponding to each element. The rhythm information indicated by these silent elements is silence. Then, all the selected silent elements are modified to be vocal elements. The rhythm information indicated by these vocal elements is vocalization.

[0117] Specifically, each element can correspond to a controlled probability, and the controlled probability of each element has the same initial value. The initial value can be pre-configured by technicians. After obtaining the above initial encoding matrix, the controlled probability of each element can be adjusted. The adjustment rules can be as follows:

[0118] Rule one, whether the element is a strong beat (the first beat) in the measure, if so, add a first preset value to the controlled probability of the element, wherein the first preset value can be configured by the technician according to actual needs.

[0119] Rule two, whether the corresponding sound production situation of the element adjacent in time is sound production, if so, add a second preset value to the controlled probability of the element, wherein the second preset value can be configured by the technician according to actual needs. The second preset value can be the same as the first preset value, or different from the first preset value.

[0120] According to the adjusted controlled probability, select a preset proportion of elements with the highest controlled probability, and modify the non-sound production elements in them to sound production elements. Wherein the preset proportion can be configured by the technician according to actual needs. In the case of the above first encoding matrix, the non-sound production element refers to the element with a value of 0, and the sound production element refers to the element with a value of 1. In the case of the above second encoding matrix, the non-sound production element refers to the element with a value of 00, and the sound production element refers to the element with a value of 01 or 10. In this case, modifying the non-sound production elements to sound production elements can be modified to 01 or 10, and can be randomly selected between the two.

[0121] In addition, if the element is selected this time, the controlled adjustment of the element is adjusted to the lowest to reduce the probability of being selected next time.

[0122] In the case of the drum point rhythm adjustment operation being a drum point sparse adjustment operation, in the encoding matrix, according to the controlled probability of each element, select part of the sound production elements, and update the part of the sound production elements to non-sound production elements. Specifically, according to the adjusted controlled probability, select a preset proportion of elements with the highest controlled probability, and modify the sound production elements in them to non-sound production elements.

[0123] In the case of the drum point rhythm adjustment operation being a drum point fast rhythm adjustment operation, the time unit is reduced, and the encoding matrix is updated according to the reduced time unit. Specifically, according to the sliding distance of the user to the drum point rhythm adjustment option, the corresponding adjustment step is determined, the current time unit is reduced by the adjustment step, and the reduced time unit is obtained. For example, the current time unit is 2s, the adjustment step is 1s, the time unit is reduced by 1s, and the reduced time unit is 1s. After reducing the time unit, the elements in the initial encoding matrix can be adjusted. For example, the current time unit is 2s, and the adjusted time unit is 1s. For any element, the element is modified to two elements, and the values of the two elements are the value of the element.

[0124] In a case where the drum beat rhythm adjustment operation is a slow drum beat rhythm adjustment operation, the time unit is increased, and the encoding matrix is updated according to the increased time unit. Specifically, the corresponding adjustment step can be determined according to the sliding distance of the user to the drum beat rhythm adjustment option, the current time unit is increased by the adjustment step, and the increased time unit is obtained. For example, the current time unit is 1s, the adjustment step is 1s, the time unit is increased by 1s, and the increased time unit is 2s. After increasing the time unit, the elements in the initial encoding matrix can be adjusted. For example, the current time unit is 1s, and the adjusted time unit is 2s. The first and second elements of the first row are merged, the value of the merged element is the value of the second element, the third and fourth elements are merged, the value of the merged element is the value of the fourth element, and so on.

[0125] Referring to Figure 7 , a schematic diagram of the controlled probability corresponding to each element in the matrix is shown, wherein the initial controlled probability of each element is 1 / N, the controlled probability of some elements is the initial controlled probability plus a preset value (σ) because they correspond to the strong beat (such as the first beat) in the measure and have not been selected and updated before, the controlled probability of some elements is the initial controlled probability plus a preset value because their adjacent elements are voiced elements, and the controlled probability of some elements is reduced to the minimum because they have been selected and updated before.

[0126] Step 105, generating drum audio based on the rhythm information of the adjusted at least one drum type.

[0127] In implementation, after adjusting the encoding matrix, the adjusted encoding matrix is decoded, that is, inverse transformed, and combined with the sampling audio corresponding to different element values to generate the corresponding audio stream data, that is, the drum audio.

[0128] All the optional technical solutions described above can be combined to form optional embodiments of the present disclosure, which will not be described one by one here.

[0129] The technical scheme provided by the embodiments of the present application can customize the audio duration of the drum audio, and then the computing device generates a drum performance instruction file corresponding to the drum audio with a specified audio duration based on the drum performance instruction generation model, wherein the drum performance instruction file includes rhythm information of at least one drum type, and the drums can include bass drums, snare drums, military drums, etc. In addition, the user can also customize the drum beat rhythm, and then the computing device can adjust the rhythm information of at least one drum type included in the drum performance instruction file according to the user's customized drum beat rhythm, and finally generate the drum audio based on the adjusted rhythm information of at least one drum type. It can be seen that in this scheme, the user can generate personalized drum audio by customizing the audio duration of the drum audio and customizing the drum beat rhythm.

[0130] Based on the same technical concept, the embodiments of the present application also provide a drum audio generation device, which can be a computing device, such as a server or a terminal. Figure 8 As shown, the device comprises a detection module 510 and a generation module 520, wherein:

[0131] The detection module 510 is configured to detect an audio duration setting operation of a user account.

[0132] The generation module 520 is configured to generate, based on a drum playing instruction generation model, a drum playing instruction file corresponding to a drum audio of a target duration, wherein the target duration matches a duration indicated by the audio duration setting operation, and the drum playing instruction file comprises rhythm information corresponding to each drum type in at least one drum type, and the rhythm information corresponding to each drum type is used to indicate a sound production condition of the corresponding drum type in each unit time.

[0133] The detection module 510 is further configured to detect a drum point rhythm adjustment operation of the user account.

[0134] The generation module 520 is further configured to adjust the rhythm information of each drum type based on the drum point rhythm adjustment operation, and generate a drum audio based on the adjusted rhythm information of each drum type.

[0135] In a possible implementation, the generation module 520 is configured to:

[0136] generate an initial encoding matrix based on the drum playing instruction file, wherein an element in the nth row and the mth column of the initial encoding matrix is used to indicate a sound production condition of the nth drum type in the mth time unit included in the drum playing instruction file;

[0137] update at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation;

[0138] generate a drum audio based on an updated encoding matrix of the initial encoding matrix.

[0139] In a possible implementation, the generation module 520 is configured to:

[0140] In a case where the drum point rhythm adjustment operation is a drum point intensive adjustment operation, in the initial encoding matrix, a part of non-sound elements are selected according to a controlled probability corresponding to each element, wherein the non-sound element indicates a non-sound production condition, and the controlled probability corresponding to each element is determined based on whether a beat corresponding to the element is a strong beat and whether an adjacent element of the element is a sound element.

[0141] The part of the non-sound elements are all modified to sound elements, wherein the sound elements indicate the sound situation as sounding;

[0142] In a case where the drum rhythm adjustment operation is a drum rhythm sparse adjustment operation, in the encoding matrix, according to the controlled probability corresponding to each element, part of the sound elements are selected;

[0143] The part of the sound elements are all updated to non-sound elements.

[0144] In a possible implementation, the generation module is configured to:

[0145] In a case where the drum rhythm adjustment operation is a drum rhythm fast rhythm adjustment operation, the time unit is reduced;

[0146] According to the reduced time unit, the encoding matrix is updated;

[0147] In a case where the drum rhythm adjustment operation is a drum rhythm slow rhythm adjustment operation, the time unit is increased;

[0148] According to the increased time unit, the elements in the encoding matrix are updated.

[0149] In a possible implementation, the detection module is further configured to:

[0150] An audio duration setting interface is displayed, wherein the audio duration setting interface includes a custom time option;

[0151] A selection operation of a user account on the custom time option is detected;

[0152] A custom time sub-interface is displayed;

[0153] An audio duration set by the user account in the custom time sub-interface is detected;

[0154] Based on the audio duration, the target duration is determined.

[0155] In a possible implementation, the detection module is further configured to:

[0156] An audio playing interface is displayed, wherein the audio playing interface includes a tuning option;

[0157] A selection operation of the user account on the tuning option is detected;

[0158] A tuning sub-interface is displayed, wherein the tuning sub-interface includes a drum rhythm adjustment option;

[0159] An adjustment operation of the user account on the drum rhythm adjustment option is detected;

[0160] The generation module is configured to:

[0161] Adjust the rhythm information of each drum type based on the adjustment operation on the drum point rhythm adjustment option.

[0162] In a possible implementation, the drum performance instruction file is a musical instrument digital interface (MIDI) file. In the technical solution provided by the embodiments of the present application, a user can customize the audio duration of drum audio, and then a computing device generates a drum performance instruction file corresponding to drum audio with a specified audio duration based on a drum performance instruction generation model, where the drum performance instruction file includes rhythm information of at least one drum type, and the drums can include bass drums, snare drums, military drums, etc. In addition, the user can also customize the drum point rhythm, and then the computing device can adjust the rhythm information of at least one drum type included in the drum performance instruction file according to the drum point rhythm customized by the user, and finally generate drum audio based on the adjusted rhythm information of at least one drum type. It can be seen that in this solution, the user can generate drum audio individually by customizing the audio duration of drum audio and customizing the drum point rhythm.

[0163] It should be noted that the drum audio generation apparatus provided by the above embodiments is only used as an example for dividing the above functional modules during drum audio generation. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the computing device is divided into different functional modules to complete all or part of the above described functions. In addition, the drum audio generation apparatus and the drum audio generation method provided by the above embodiments belong to the same concept, and the specific implementation process is described in detail in the method embodiments, which will not be repeated here.

[0164] Figure 9 A structure block diagram of a terminal 600 provided by an example embodiment of the present application is shown. The terminal 600 can be a portable mobile terminal, such as a smart phone, a tablet computer, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a notebook computer, or a desktop computer. The terminal 600 can also be referred to as a user equipment, a portable terminal, a laptop terminal, a desktop terminal, or other names.

[0165] Generally, the terminal 600 includes a processor 601 and a memory 602.

[0166] The processor 601 can include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 601 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 601 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 601 can be integrated with a GPU (Graphics Processing Unit) for rendering and drawing content required to be displayed by the display screen. In some embodiments, the processor 601 can further include an AI (Artificial Intelligence) processor for processing computing operations related to machine learning.

[0167] The memory 602 can include one or more computer-readable storage media that can be non-transitory. The memory 602 can also include a high-speed random access memory, and a nonvolatile memory such as one or more disk storage devices, flash storage devices.

[0168] In some embodiments, the terminal 600 can also optionally include a peripheral device interface 603 and at least one peripheral device. The processor 601, the memory 602, and the peripheral device interface 603 can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface 603 through a bus, a signal line, or a circuit board. Specifically, the peripheral device includes at least one of a radio frequency circuit 604, a display screen 605, a camera assembly 606, an audio circuit 607, a positioning assembly 608, and a power supply 609.

[0169] The peripheral device interface 603 can be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 601 and the memory 602. In some embodiments, the processor 601, the memory 602, and the peripheral device interface 603 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 601, the memory 602, and the peripheral device interface 603 can be implemented on a separate chip or circuit board, and the present embodiment does not limit this.

[0170] The radio frequency circuit 604 is configured to receive and send RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 604 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 604 converts electrical signals to electromagnetic signals for transmission, or vice versa. Optionally, the radio frequency circuit 604 includes an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a subscriber identity module card, and the like. The radio frequency circuit 604 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to, the World Wide Web, a metropolitan area network, an intranet, various generations of mobile communication networks (2G, 3G, 4G, and 5G), a wireless local area network, and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 604 can also include NFC (Near Field Communication) related circuitry, which is not limited in the present application.

[0171] The display screen 605 is configured to display a UI (User Interface). The UI can include graphics, text, icons, video, and any combination thereof. When the display screen 605 is a touch display screen, the display screen 605 also has the ability to collect touch signals on or above the surface of the display screen 605. The touch signals can be input as control signals to the processor 601 for processing. At this time, the display screen 605 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 605 can be one, arranged on the front panel of the terminal 600; in other embodiments, the display screen 605 can be at least two, arranged on different surfaces of the terminal 600 or in a folding design; in other embodiments, the display screen 605 can be a flexible display screen, arranged on a curved surface or a folding surface of the terminal 600. Even, the display screen 605 can also be arranged in an irregular shape, that is, a special-shaped screen. The display screen 605 can be made of materials such as LCD (Liquid Crystal Display) and OLED (Organic Light-Emitting Diode).

[0172] The camera component 606 is configured to capture images or videos. Optionally, the camera component 606 includes a front-facing camera and a rear-facing camera. Generally, the front-facing camera is disposed on the front panel of the terminal, and the rear-facing camera is disposed on the back of the terminal. In some embodiments, the rear-facing camera is at least two, which are any one of a main camera, a depth-of-field camera, a wide-angle camera, and a telephoto camera, to realize the background blur function by fusing the main camera and the depth-of-field camera, the panorama shooting and VR (Virtual Reality) shooting function by fusing the main camera and the wide-angle camera, or other fusion shooting functions. In some embodiments, the camera component 606 can further include a flash. The flash can be a single-color-temperature flash or a dual-color-temperature flash. The dual-color-temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0173] The audio circuit 607 can include a microphone and a speaker. The microphone is configured to capture sound waves of a user and an environment, and convert the sound waves into an electrical signal input to the processor 601 for processing or to the radio frequency circuit 604 to realize voice communication. For the purpose of stereo sound collection or noise reduction, the microphone can be multiple, which are respectively disposed at different parts of the terminal 600. The microphone can also be an array microphone or an omnidirectional collection microphone. The speaker is configured to convert an electrical signal from the processor 601 or the radio frequency circuit 604 into sound waves. The speaker can be a traditional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert an electrical signal into a sound wave audible to humans, but also convert an electrical signal into an inaudible sound wave to humans for ranging purposes. In some embodiments, the audio circuit 607 can further include a headphone jack.

[0174] The positioning component 608 is configured to locate the current geographical position of the terminal 600 to realize navigation or LBS (Location Based Service). The positioning component 608 can be a positioning component of a GPS (Global Positioning System), a Beidou system, or a Galileo system.

[0175] The power supply 609 is configured to supply power to each component in the terminal 600. The power supply 609 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 609 includes a rechargeable battery, the rechargeable battery can be a wired charging battery or a wireless charging battery. The wired charging battery is a battery charged through a wired line, and the wireless charging battery is a battery charged through a wireless coil. The rechargeable battery can also be used to support fast charging technology.

[0176] In some embodiments, the terminal 600 further comprises one or more sensors 610. The one or more sensors 610 include, but are not limited to, an acceleration sensor 611, a gyroscope sensor 612, a pressure sensor 613, a fingerprint sensor 614, an optical sensor 615, and a proximity sensor 616.

[0177] The acceleration sensor 611 can detect the acceleration magnitude in three coordinate axes of a coordinate system established by the terminal 600. For example, the acceleration sensor 611 can be used to detect the components of the gravitational acceleration in three coordinate axes. The processor 601 can control the display screen 605 to display the user interface in a landscape view or a portrait view according to the gravitational acceleration signals collected by the acceleration sensor 611. The acceleration sensor 611 can also be used for game or user motion data collection.

[0178] The gyroscope sensor 612 can detect the body orientation and rotation angle of the terminal 600. The gyroscope sensor 612 can work with the acceleration sensor 611 to collect the 3D motion of the user to the terminal 600. The processor 601 can implement the following functions according to the data collected by the gyroscope sensor 612: motion sensing (e.g., changing the UI according to the user's tilt operation), image stabilization when shooting, game control, and inertial navigation.

[0179] The pressure sensor 613 can be disposed on the side frame of the terminal 600 and / or the lower layer of the display screen 605. When the pressure sensor 613 is disposed on the side frame of the terminal 600, the user's holding signal to the terminal 600 can be detected, and the left-hand or right-hand recognition or shortcut operation can be performed by the processor 601 according to the holding signal collected by the pressure sensor 613. When the pressure sensor 613 is disposed on the lower layer of the display screen 605, the controllable control on the UI interface can be controlled by the processor 601 according to the user's pressure operation to the display screen 605. The controllable control includes at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0180] The fingerprint sensor 614 is used to collect the user's fingerprint. The identity of the user can be recognized by the processor 601 according to the fingerprint collected by the fingerprint sensor 614, or by the fingerprint sensor 614 according to the collected fingerprint. When the identity of the user is recognized as a trusted identity, the processor 601 authorizes the user to perform related sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, payment, and changing settings, etc. The fingerprint sensor 614 can be disposed on the front, back, or side of the terminal 600. When the terminal 600 is provided with a physical button or a manufacturer's logo, the fingerprint sensor 614 can be integrated with the physical button or the manufacturer's logo.

[0181] The optical sensor 615 is configured to collect ambient light intensity. In one embodiment, the processor 601 can control the display brightness of the display screen 605 according to the ambient light intensity collected by the optical sensor 615. Specifically, when the ambient light intensity is high, the display brightness of the display screen 605 is increased; when the ambient light intensity is low, the display brightness of the display screen 605 is decreased. In another embodiment, the processor 601 can also dynamically adjust the shooting parameters of the camera assembly 606 according to the ambient light intensity collected by the optical sensor 615.

[0182] The proximity sensor 616, also referred to as a distance sensor, is usually arranged on the front panel of the terminal 600. The proximity sensor 616 is configured to collect the distance between the user and the front of the terminal 600. In one embodiment, when the proximity sensor 616 detects that the distance between the user and the front of the terminal 600 gradually decreases, the display screen 605 is switched from the on-screen state to the off-screen state by the processor 601; when the proximity sensor 616 detects that the distance between the user and the front of the terminal 600 gradually increases, the display screen 605 is switched from the off-screen state to the on-screen state by the processor 601.

[0183] Those skilled in the art can understand that the structure shown in FIG. 6 is not a limitation on the terminal 600, and the terminal 600 can include more or fewer components than those shown in the figure, or combine certain components, or adopt a different arrangement of components. Figure 9

[0184] Figure 10 FIG. 10 is a structural schematic diagram of a server provided by an embodiment of the present application. The server 1000 can have great differences due to different configurations or performances, and can include one or more central processing units (CPUs) 1001 and one or more memories 1002. The memory 1002 stores at least one instruction, which is loaded and executed by the processor 1001 to implement the method provided by each method embodiment. Of course, the computing device can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for implementing device functions, and the like. The computing device can also include other components for implementing device functions, which are not described here.

[0185] ​In an example embodiment, a computer readable storage medium, such as a memory including instructions executable by a processor in a terminal to perform the method of training the neural network model in the above-described embodiments, is also provided. The computer readable storage medium can be non-transitory. For example, the computer readable storage medium can be a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0186] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data need to comply with relevant laws, regulations, and standards in relevant countries and regions. For example, the audio duration setting operation, the drum beat rhythm adjustment operation, etc. involved in the present application are obtained under full authorization.

[0187] A person of ordinary skill in the art can understand that all or part of the steps of the above-described embodiments can be completed by hardware, or by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium. The storage medium mentioned above can be a read-only memory, a magnetic disk, or an optical disk, etc.

[0188] The above description is only optional embodiments of the present application and does not limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method of drum sound generation, characterized by, The method comprises: detecting an audio duration setting operation of a user account; generating, based on a drum playing instruction generation model, a drum playing instruction file corresponding to drum audio of a target duration, wherein the target duration matches a duration indicated by the audio duration setting operation, and the drum playing instruction file comprises rhythm information corresponding to each drum type of at least one drum type, and the rhythm information corresponding to each drum type is used to indicate the sound condition of the corresponding drum type in each unit time; detecting a drum point rhythm adjustment operation of the user account; generating an initial encoding matrix based on the drum playing instruction file, wherein an element in the nth row and the mth column of the initial encoding matrix is used to indicate the sound condition of the nth drum type in the mth time unit included in the drum playing instruction file; updating at least one element in the initial encoding matrix based on the drum point rhythm adjustment operation; generating drum audio based on the updated encoding matrix of the initial encoding matrix.

2. The method of claim 1, wherein, The method further comprises: in a case where the drum point rhythm adjustment operation is a drum point dense adjustment operation, selecting, in the initial encoding matrix, part of non-sound elements according to a controlled probability corresponding to each element, wherein the non-sound element indicates a non-sound condition, and the controlled probability corresponding to each element is determined based on whether the beat corresponding to each element is a strong beat and whether the adjacent element of each element is a sound element; modifying the part of non-sound elements to sound elements, wherein the sound element indicates a sound condition; in a case where the drum point rhythm adjustment operation is a drum point sparse adjustment operation, selecting, in the encoding matrix, part of sound elements according to a controlled probability corresponding to each element; updating the part of sound elements to non-sound elements.

3. The method of claim 1, wherein, The method further comprises: in a case where the drum point rhythm adjustment operation is a drum point fast rhythm adjustment operation, reducing the time unit; updating the encoding matrix according to the reduced time unit; in a case where the drum point rhythm adjustment operation is a drum point slow rhythm adjustment operation, increasing the time unit; updating the elements in the encoding matrix according to the increased time unit.

4. The method of claim 1, wherein, The method further comprises: displaying an audio duration setting interface, wherein the audio duration setting interface comprises a custom time option; The method further comprises: detecting a selection operation of the custom time option by the user account; displaying a custom time sub-interface; detecting an audio duration set by the user account in the custom time sub-interface; determining the target duration based on the audio duration.

5. The method of claim 1, wherein, The method further comprises: displaying an audio playing interface, wherein the audio playing interface comprises a tuning option; The method further comprises: detecting a selection operation of the tuning option by the user account; displaying a tuning sub-interface, wherein the tuning sub-interface comprises a drum rhythm adjustment option; detecting an adjustment operation of the user account on the drum rhythm adjustment option; adjusting the rhythm information of each drum type based on the drum rhythm adjustment operation, comprising: adjusting the rhythm information of each drum type based on the adjustment operation on the drum rhythm adjustment option.

6. The method according to any one of claims 1-5, characterized in that, The drum performance instruction file is a musical instrument digital interface (MIDI) file.

7. A computing device, comprising: The computing device comprises a processor and a memory, and the memory stores at least one instruction which is loaded and executed by the processor to implement the operations performed by the drum audio generation method according to any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction which is loaded and executed by the processor to implement the operations performed by the drum audio generation method according to any one of claims 1 to 6.

9. A computer program product, characterised in that, The computer program product stores at least one instruction which is loaded and executed by the processor to implement the operations performed by the drum audio generation method according to any one of claims 1 to 6.

Images