Method and device for generating arrangement audio, storage medium and program product

Abstract

The invention provides an arrangement audio generation method and device, a storage medium and a program product, and belongs to the technical field of audio. The method comprises the following steps: acquiring at least one musical instrument audio characteristic parameter and at least one noise audio characteristic parameter; generating a fundamental tone signal of a target musical instrument based on the at least one musical instrument audio characteristic parameter; generating a noise signal of the target musical instrument based on the at least one noise audio characteristic parameter; based on a target synthesis control parameter, performing synthesis processing on the gene signal and the noise signal to obtain a musical instrument audio signal of the target musical instrument, which accords with a target tone; and generating a target arrangement audio based on the musical instrument audio signal. By adopting the method and the device, various target arrangement audios which can be dynamically adjusted are realized, and the arrangement effect is effectively improved.

Description

Technical Field

[0001] This disclosure relates to the field of audio technology, and in particular to a method, apparatus, storage medium, and program product for generating arranged audio. Background Technology

[0002] To improve the user experience, some music applications are becoming increasingly feature-rich, including music arrangement functions. Users can add a segment of their desired instrument at a specific point in the initial music arrangement audio to generate a new music arrangement audio.

[0003] The current method for generating new arrangement audio is to first perform various scenes (i.e., different instruments, different rhythms and dynamics) on physical instruments to obtain multiple audio recordings of the instrument's performance, and then save them in a music library. In this way, users can select the desired audio from these multiple audio recordings through the music application and add it to the initial arrangement audio.

[0004] However, the number of audio files stored in the music library is limited and cannot cover all performance scenarios, resulting in the selection of audio files from the music library failing to meet the user's needs, which in turn leads to poor arrangement quality. Summary of the Invention

[0005] This disclosure provides a method for generating arranged audio, which can solve the technical problems existing in related technologies. The technical solution is as follows: Firstly, a method for generating arranged audio is provided, the method comprising: Obtain at least one instrument audio feature parameter and at least one noise audio feature parameter; Based on the at least one musical instrument audio feature parameter, generate the fundamental tone signal of the target musical instrument; Based on the at least one noise audio feature parameter, generate the noise signal of the target musical instrument; Based on the target synthesis control parameters, the gene signal and the noise signal are synthesized to obtain the instrument audio signal that matches the target timbre of the target instrument; Based on the instrument audio signal, the target arrangement audio is generated.

[0006] In one alternative approach, generating the fundamental tone signal of the target instrument based on the at least one instrument audio feature parameter includes: The at least one instrument audio characteristic parameter is input into the oscillator function to generate the fundamental tone signal of the target instrument.

[0007] In one alternative embodiment, the oscillator function includes a waveform function and a first amplitude envelope function, and the instrument audio characteristic parameters include waveform adjustment parameters and first amplitude envelope adjustment parameters; The step of inputting the at least one instrument audio feature parameter into an oscillator function to obtain the fundamental tone signal of the target instrument includes: The waveform adjustment parameters are input into the waveform function to generate a waveform signal; Input the first amplitude envelope adjustment parameter into the first amplitude envelope function to obtain the adjusted first amplitude envelope function; Based on the adjusted first amplitude envelope function and the waveform signal, the fundamental tone signal of the target musical instrument is generated.

[0008] In one alternative embodiment, the instrument audio characteristic parameters further include a first note velocity parameter; The step of generating the fundamental tone signal of the target musical instrument based on the adjusted first amplitude envelope function and the waveform signal includes: Based on the adjusted first amplitude envelope function, the waveform signal is subjected to amplitude envelope modulation to obtain the modulated waveform signal. Based on the first note velocity parameter, the modulated waveform signal is subjected to overall amplitude modulation to obtain the fundamental tone signal of the target instrument.

[0009] In one alternative embodiment, the waveform function includes a pitch envelope function and a low-frequency oscillation function, and the waveform adjustment parameters include a fundamental frequency parameter, a pitch envelope adjustment parameter, and an oscillation adjustment parameter. The step of inputting the waveform adjustment parameters into the waveform function to generate a waveform signal includes: The pitch envelope adjustment parameters are input into the pitch envelope function to obtain the adjusted pitch envelope function; The oscillation adjustment parameters are input into the low-frequency oscillation function to obtain the adjusted low-frequency oscillation function; The waveform signal is generated based on the fundamental frequency parameters, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function.

[0010] In one alternative approach, generating the fundamental tone signal of the target instrument based on the adjusted first amplitude envelope function and the waveform signal includes: Based on formula (1), the fundamental tone signal of the target musical instrument is generated: Formula (1) in, For sampling point index, The fundamental tone signal, The adjusted first amplitude envelope function, The waveform signal, The fundamental frequency parameter, The adjusted pitch envelope function, This is the adjusted low-frequency oscillation function.

[0011] In one alternative approach, generating the noise signal of the target instrument based on the at least one noise audio feature parameter includes: The noise signal of the target musical instrument is generated by inputting the at least one noise audio feature parameter into the noise function.

[0012] In one alternative embodiment, the noise function includes a filter and a second amplitude envelope function, and the noise audio characteristic parameters include filter parameters and a second amplitude envelope adjustment parameter; The step of inputting the at least one noise audio feature parameter into a noise function to generate a noise signal for the target instrument includes: The filter parameters are input into the filter to generate an initial noise signal; Input the second amplitude envelope adjustment parameter into the second amplitude envelope function to obtain the adjusted second amplitude envelope function; Based on the adjusted second amplitude envelope function and the initial noise signal, the noise signal of the target musical instrument is generated.

[0013] In one alternative embodiment, the noise audio characteristic parameters further include a second note velocity parameter; The step of generating the noise signal for the target musical instrument based on the adjusted second amplitude envelope function and the initial noise signal includes: Based on the adjusted second amplitude envelope function, the initial noise signal is subjected to amplitude envelope modulation to obtain the modulated initial noise signal; Based on the second note velocity parameter, the modulated initial noise signal is subjected to overall amplitude modulation to obtain the noise signal of the target instrument.

[0014] In one alternative approach, the filter parameters include at least one of noise type, filter cutoff frequency, and filter type.

[0015] In one alternative approach, generating the noise signal of the target instrument based on the adjusted second amplitude envelope function and the initial noise signal includes: Based on formula (2), the noise signal of the target musical instrument is generated: Formula (2) in, The noise signal, The adjusted second amplitude envelope function, For the filter, For the aforementioned noise type, The cutoff frequency of the filter is... This refers to the filter type.

[0016] In one optional approach, the genetic signal and the noise signal are synthesized based on target synthesis control parameters to obtain an instrument audio signal that matches the target timbre for the target instrument, including: Based on the target synthesis control parameters, the pitch signal and the noise signal are weighted and processed to obtain the instrument audio signal.

[0017] In one alternative approach, the method further includes: Obtain the distortion intensity parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the distortion intensity parameter, the instrument audio signal is distorted to obtain the target arranged audio.

[0018] In one alternative approach, the method further includes: Obtain the initial arrangement audio; The step of generating the target arrangement audio based on the instrument audio signal includes: The initial arrangement audio and the instrument audio signal are synthesized to obtain the target arrangement audio.

[0019] In one alternative approach, the method further includes: Obtain audio-visual parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the sound image parameters and the instrument audio signal, generate left channel instrument audio and right channel instrument audio; The target arrangement audio is generated based on the left channel instrument audio and the right channel instrument audio.

[0020] In an alternative approach, the method further includes: Get global volume parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the global volume parameters, the global volume of the instrument audio signal is adjusted to obtain the adjusted instrument audio signal; The target arrangement audio is generated based on the adjusted instrument audio signal.

[0021] Secondly, an apparatus for generating arranged audio is provided, the apparatus comprising: The acquisition module is used to acquire at least one instrument audio feature parameter and at least one noise audio feature parameter; A pitch generation module is used to generate a pitch signal of a target instrument based on the at least one instrument audio feature parameter; A noise generation module is used to generate a noise signal for the target musical instrument based on the at least one noise audio feature parameter. The synthesis module is used to synthesize the gene signal and the noise signal based on the target synthesis control parameters to obtain the instrument audio signal that matches the target timbre of the target instrument; The arrangement generation module is used to generate target arrangement audio based on the instrument audio signal.

[0022] Thirdly, a computer device is provided, the computer device including a processor and a memory, the memory storing at least one instruction, the instruction being loaded and executed by the processor to implement the operation performed by the method for generating arranged audio.

[0023] Fourthly, a computer-readable storage medium is provided, wherein at least one instruction is stored in the storage medium, the instruction being loaded and executed by a processor to implement the operation performed by the method for generating arranged audio.

[0024] Fifthly, a computer program product is provided, the computer program product including at least one instruction, the at least one instruction being loaded and executed by a processor to implement the operations performed by the method for generating arranged audio.

[0025] The beneficial effects of the technical solution provided in this disclosure are as follows: the solution mentioned in this disclosure can adjust the instrument audio characteristic parameters and noise audio characteristic parameters according to different scenario requirements of the target arranged audio, thereby realizing a variety of target arranged audio that can be dynamically adjusted, effectively improving the arrangement effect. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this disclosure, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a flowchart illustrating a method for generating arranged audio according to an embodiment of this disclosure; Figure 2 This is a flowchart illustrating a method for generating arranged audio according to an embodiment of this disclosure; Figure 3This is a schematic diagram of the structure of an audio arrangement generation device provided in an embodiment of this disclosure; Figure 4 This is a structural block diagram of a terminal provided in an embodiment of this disclosure; Figure 5 This is a structural block diagram of a server provided in an embodiment of this disclosure. Detailed Implementation

[0028] To make the objectives, technical solutions, and advantages of this disclosure clearer, the embodiments of this disclosure will be described in further detail below with reference to the accompanying drawings.

[0029] In many current music applications, users can compose music according to their own preferences and needs to obtain an arranged audio track. For example, a user can generate an audio track played by a target instrument within the music application, or a user can first input an audio file into the music application and then add the audio played by the target instrument at a target time point in the audio file, and so on.

[0030] Current methods for generating arrangement audio involve pre-playing various scenarios (different instruments, rhythms, and dynamics) on physical instruments to obtain multiple audio samples for each instrument. These samples are then stored in a music library. Users can then select the desired audio from these samples using a music application and add it to their initial arrangement. However, the number of audio samples stored in the library is limited and cannot cover all the scenarios required by users, thus reducing the quality of the arrangement. Furthermore, the audio samples stored in the library consume significant storage space, resulting in high storage costs.

[0031] Based on this, the present disclosure provides a method for generating arranged audio. In this method, the instrument audio feature parameters and noise audio feature parameters can be adjusted according to different scenario requirements of the target arranged audio, thereby realizing dynamically adjustable and diverse target arranged audio, effectively improving the arrangement effect.

[0032] In this embodiment of the disclosure, the execution entity of the method for generating arranged audio is an audio generation device. Optionally, the device is a hardware device, such as a computer device, which is a terminal or server, including but not limited to mobile phones, laptops, or desktop computers. Optionally, the device is a software device, such as a software program installed on a computer device.

[0033] In this embodiment of the disclosure, the generated instrument audio signal can be an audio signal corresponding to various instruments, such as a drum sound audio signal corresponding to various drum instruments, etc.

[0034] The following section uses the drum kit as an example to illustrate the process of generating arranged audio. (See [link to documentation]). Figure 1 Steps S101 to S105.

[0035] Step S101: Obtain at least one instrument audio feature parameter and at least one noise audio feature parameter.

[0036] Among them, the instrument audio characteristic parameters are adjustable parameters used to generate the fundamental tone signal of the target instrument. Different instrument audio characteristic parameters can result in different timbre and volume of the generated fundamental tone signal. Similarly, the noise audio characteristic parameters are adjustable parameters used to generate the noise signal of the target instrument. Different noise audio characteristic parameters can result in different timbre and volume of the generated noise signal.

[0037] In this embodiment of the disclosure, a music application is installed on the terminal, and the user can open the music application on the terminal and enter the music arrangement page.

[0038] Users can choose to directly generate a new arranged audio on the arrangement page, or they can input an initial arranged audio into the music application, that is, input an already arranged audio (for example, an already arranged MIDI (Musical Instrument Digital Interface) file). Then, they can input at least one instrument audio feature parameter and at least one noise audio feature parameter at a target time point in this new arranged audio or the initial arranged audio, so that the terminal can obtain these at least one instrument audio feature parameter and at least one noise audio feature parameter.

[0039] The differences in instrument audio characteristic parameters and noise audio characteristic parameters allow music applications to output corresponding audio, enabling users to adjust these parameters.

[0040] The number of instrument audio feature parameters and noise audio feature parameters can both be one or more. If some instrument audio feature parameters or noise audio feature parameters are not entered, their default values ​​can be obtained for subsequent audio generation.

[0041] Step S102: Generate the pitch signal of the target instrument based on at least one instrument audio feature parameter.

[0042] In this embodiment of the disclosure, after the terminal obtains at least one musical instrument audio feature parameter, the fundamental tone signal of the target musical instrument corresponding to the at least one musical instrument audio feature parameter can be generated based on the at least one musical instrument audio feature parameter.

[0043] In one alternative approach, the method for generating the pitch signal of the target instrument based on the instrument's audio feature parameters can be: inputting at least one instrument's audio feature parameter into an oscillator function to generate the pitch signal of the target instrument.

[0044] Music applications store oscillator functions, which include multiple fixed and adjustable parameters. These adjustable parameters are the instrument audio characteristic parameters input by the user. By adjusting the instrument audio characteristic parameters, the user can change the timbre and volume of the target instrument's fundamental tone signal generated by the oscillator function.

[0045] Once at least one instrument audio feature parameter is obtained, it is input into a preset oscillator function to generate the corresponding fundamental tone signal of the target instrument.

[0046] Optionally, the oscillator function may include a waveform function and a first amplitude envelope function, and the instrument audio characteristic parameters may include waveform adjustment parameters and first amplitude envelope adjustment parameters.

[0047] Correspondingly, the method for generating the fundamental tone signal of the target instrument can be as follows: input the waveform adjustment parameters into the waveform function to generate the waveform signal; input the first amplitude envelope adjustment parameters into the first amplitude envelope function to obtain the adjusted first amplitude envelope function; and generate the fundamental tone signal of the target instrument based on the adjusted first amplitude envelope function and the waveform signal.

[0048] The corresponding formula can be expressed as follows: Formula (3) in, For sampling point index, For pitch signal, The adjusted first amplitude envelope function, It is a waveform signal.

[0049] Correspondingly, the waveform adjustment parameters are the adjustable parameters assigned in the waveform function, and the first amplitude envelope adjustment parameters are the adjustable parameters assigned in the first amplitude envelope function.

[0050] In this way, after generating the waveform signal based on the waveform adjustment parameters and waveform function, the amplitude envelope of the waveform signal is adjusted based on the adjusted first amplitude envelope function, so that the obtained fundamental tone signal of the target instrument is more in line with the user's needs.

[0051] The first amplitude envelope function can be any reasonable function, such as the AD (Attack Decay) amplitude envelope function, and the corresponding formula is as follows: Formula (4) in, For sampling point index, The adjusted first amplitude envelope function, It can be 1, The normalization coefficient is... and This belongs to the first amplitude envelope adjustment parameter, where, For the first attack time, This is the first decay time.

[0052] Users can adjust the first amplitude envelope function by adjusting the first attack time and the first decay time, thereby adjusting the pitch signal.

[0053] Optionally, taking a drum as an example, The value can range from 0.1 to 10 ms. The value range can be 50~500ms, and of course, it can also be any other reasonable value. This disclosure does not limit it.

[0054] Alternatively, the first amplitude envelope function can also be an ADRS (Attack-Decay-Sustain-Release) amplitude envelope function, which is equivalent to adding two first amplitude envelope adjustment parameters—first sustain time and first release time—to the first attack time and first decay time. Users can adjust the adjusted first amplitude envelope function by adjusting the first attack time, first decay time, first sustain time, and first release time, thereby adjusting the fundamental tone signal of the target instrument.

[0055] Among them, the AD amplitude envelope function is suitable for generating short drum beats such as bass drum, snare drum, and tom-tom, while the ADRS amplitude envelope function is more suitable for generating long sustained drum beats such as cymbals. It can be set according to actual needs. In this embodiment, the type of the first amplitude envelope function is not specifically limited.

[0056] Optionally, the instrument audio feature parameters may also include the first note velocity parameter. The corresponding processing is as follows: Based on the adjusted first amplitude envelope function, amplitude envelope modulation is performed on the waveform signal to obtain the modulated waveform signal. Based on the first note velocity parameter, overall amplitude modulation is performed on the modulated waveform signal to obtain the fundamental tone signal of the target instrument.

[0057] The corresponding formula can be expressed as follows: Formula (5) in, This is the dynamics parameter for the first note.

[0058] After obtaining the modulated waveform signal, the first note velocity parameter can be used to modulate the overall amplitude of the modulated waveform signal, thereby adjusting the overall amplitude of the fundamental tone signal of the target instrument, which is equivalent to adjusting the velocity when playing a drum instrument to a certain extent.

[0059] In this way, the fundamental tone signal of the generated target instrument is adjusted by means of waveform adjustment parameters, first amplitude envelope adjustment parameters, and first note velocity parameters to meet the user's needs.

[0060] Optionally, waveform adjustment parameters may include waveform type parameters, such as square wave, sawtooth, sine, etc.

[0061] Optionally, the oscillator function includes a pitch envelope function and a low-frequency oscillation function, and the waveform adjustment parameters include a fundamental frequency parameter, a pitch envelope adjustment parameter, and an oscillation adjustment parameter.

[0062] Correspondingly, the waveform signal generation process can be as follows: Input the pitch envelope adjustment parameter into the pitch envelope function to obtain the adjusted pitch envelope function. Input the oscillation adjustment parameter into the low-frequency oscillation function to obtain the adjusted low-frequency oscillation function. Based on the fundamental frequency parameter, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function, generate the waveform signal.

[0063] In this way, the fundamental tone signal of the generated target instrument is adjusted by using the fundamental frequency parameter, pitch envelope adjustment parameter, oscillation adjustment parameter, first amplitude envelope adjustment parameter, and first note velocity parameter to meet the user's needs.

[0064] Alternatively, the waveform signal processing method based on the fundamental frequency parameters, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function can be as follows: Based on formula (1), the fundamental tone signal of the target instrument is generated: Formula (1) in, For sampling point index, For pitch signal, The adjusted first amplitude envelope function, It is a waveform signal. As a fundamental frequency parameter, it can be used to control the fundamental pitch of the drum beats. This is the adjusted pitch envelope function. This is the adjusted low-frequency oscillation function.

[0065] The pitch envelope function can be any reasonable function, such as the AD amplitude envelope function, and the corresponding formula is as follows: Formula (6) in, For sampling point index, The adjusted pitch envelope function, It can be 1, The normalization coefficient is... and This belongs to the pitch envelope adjustment parameters, among which, This is the second attack time. This is the second decay time.

[0066] Users can adjust the pitch envelope function by adjusting the second attack time and the second decay time, thereby adjusting the pitch signal.

[0067] Of course, the pitch envelope function can also be other envelope functions such as the ADRS amplitude envelope function. This is equivalent to adding two pitch envelope adjustment parameters—the second hold time and the second release time—to the second attack time and the second decay time. Users can adjust the adjusted pitch envelope function by adjusting the second attack time, the second decay time, the second hold time, and the second release time, thereby adjusting the fundamental tone signal.

[0068] The low-frequency oscillation function can be any reasonable function; for example, its corresponding formula can be as follows: Formula (7) in, For sampling point index, This refers to the adjusted low-frequency oscillation function. Sampling rate, and Here, is the low-frequency oscillation coefficient, where, For modulation depth, For frequency.

[0069] Users can adjust the low-frequency oscillation function by adjusting the sampling rate, modulation depth, and frequency, thereby adjusting the pitch signal.

[0070] The modulation depth can be 0 to 1, and the frequency can be 0.1 to 5 Hz. Of course, it can also be any other reasonable value. This disclosure does not limit the values.

[0071] In this embodiment of the disclosure, the low-frequency oscillation envelope function can also be any other reasonable function, and this embodiment of the disclosure does not limit it.

[0072] Based on the example above, users can adjust the basic frequency parameter to 80Hz, the waveform type parameter to sawtooth, and the first attack time to 5ms. In this way, the first amplitude envelope function rises rapidly to the peak, generating a bass drum basic tone with a strong impact.

[0073] Optionally, if the instrument's audio characteristic parameters also include the first note velocity parameter, the processing method for generating the fundamental tone signal of the target instrument based on the fundamental frequency parameter, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function can be as follows: Based on formula (8), the fundamental tone signal of the target instrument is generated: Formula (8) According to the above formula (8), the user can adjust the fundamental tone signal of the generated target instrument by inputting waveform type parameters, fundamental frequency parameters, pitch envelope adjustment parameters, oscillation adjustment parameters, first amplitude envelope adjustment parameters and first note velocity parameters, so as to meet the user's needs.

[0074] It is understood that the above are only examples of several possible oscillator functions, and the oscillator function can also be any other reasonable function. This disclosure does not specifically limit the embodiments in this regard.

[0075] In another alternative approach, at least one instrument audio feature parameter can be input into the pitch generation model to obtain the output pitch signal of the target instrument.

[0076] The pitch generation model can be any pre-trained machine learning model. After obtaining at least one instrument audio feature parameter, it can be input into the pitch generation model to obtain the output pitch signal of the target instrument.

[0077] Step S103: Generate the noise signal of the target instrument based on at least one noise audio feature parameter.

[0078] In this embodiment of the disclosure, after the terminal obtains at least one noise audio feature parameter, a noise signal corresponding to the noise audio feature parameter can be generated based on the at least one noise audio feature parameter.

[0079] In one alternative approach, the method for generating a noise signal based on noise audio feature parameters can be: inputting at least one noise audio feature parameter into a noise function to generate a noise signal for the target instrument.

[0080] Music applications store a noise function, which includes multiple fixed and adjustable parameters. These adjustable parameters are the noise audio feature parameters input by the user. Users can adjust the noise audio feature parameters to change the timbre and volume of the noise signal generated by the noise function.

[0081] Once at least one noise audio feature parameter is obtained, it is input into a preset noise function to generate the corresponding noise signal of the target instrument.

[0082] Optionally, the noise function may include a filter and a second amplitude envelope function, and the noise audio characteristic parameters may include filter parameters and a second amplitude envelope adjustment parameter.

[0083] Correspondingly, the method for generating the noise signal can be as follows: input the filter parameters into the filter to generate the initial noise signal; input the second amplitude envelope adjustment parameters into the second amplitude envelope function to obtain the adjusted second amplitude envelope function; and generate the noise signal of the target instrument based on the adjusted second amplitude envelope function and the initial noise signal.

[0084] The corresponding formula can be expressed as follows: Formula (9) in, For sampling point index, This is a noise signal. The adjusted second amplitude envelope function, For filters.

[0085] Correspondingly, the filter parameters are the adjustable parameters assigned in the filter, and the second amplitude envelope adjustment parameters are the adjustable parameters assigned in the second amplitude envelope function.

[0086] In this way, after generating the initial noise signal based on the filter parameters and the filter, the amplitude envelope of the initial noise signal is adjusted based on the adjusted second amplitude envelope function, so that the obtained noise signal can better meet the user's needs.

[0087] The second amplitude envelope function can be any reasonable function, such as the AD amplitude envelope function, and the corresponding formula is as follows: Formula (10) in, For sampling point index, The adjusted second amplitude envelope function, It can be 1, The normalization coefficient is... and This belongs to the second amplitude envelope adjustment parameter, where, This is the third attack time. This is the third decay time.

[0088] Users can adjust the second amplitude envelope function by adjusting the third attack time and the third decay time, thereby adjusting the noise audio.

[0089] Optionally, taking the generation of drum sounds as an example, The value range can be 0~1ms. The value range can be 10~200ms, and of course, it can also be any other reasonable value. This disclosure does not limit it.

[0090] Of course, the second amplitude envelope function can also be other envelope functions such as the ADRS amplitude envelope function. This is equivalent to adding two second amplitude envelope adjustment parameters—the third hold time and the third release time—to the third attack time and the third decay time. Users can adjust the adjusted second amplitude envelope function by adjusting the third attack time, the third decay time, the third hold time, and the third release time, thereby adjusting the noise signal.

[0091] Optionally, the noise audio feature parameters also include a second note velocity parameter. The corresponding processing is as follows: Based on the adjusted second amplitude envelope function, the initial noise signal is amplitude envelope modulated to obtain the modulated initial noise signal. Based on the second note velocity parameter, the modulated initial noise signal is then subjected to overall amplitude modulation to obtain the noise signal of the target instrument.

[0092] The corresponding formula can be expressed as follows: Formula (11) in, This is the dynamics parameter for the second note.

[0093] After obtaining the modulated initial noise signal, the second note velocity parameter can be used to modulate the overall amplitude of the modulated initial noise signal, thereby adjusting the overall amplitude of the generated noise audio, which is equivalent to adjusting the velocity when playing a musical instrument to a certain extent.

[0094] In this way, the generated noise signal can be adjusted by using filter parameters, second amplitude envelope adjustment parameters, and second note velocity parameters to meet user requirements.

[0095] Optionally, filter parameters may include at least one of noise type, filter cutoff frequency, and filter type.

[0096] For example, the noise signal of the target musical instrument can be generated based on formula (2): Formula (2) in, This is a noise signal. The adjusted second amplitude envelope function, For filters, Noise type, This is the filter cutoff frequency. This refers to the filter type.

[0097] In this way, the generated noise signal can be adjusted by using noise type, filter cutoff frequency, filter type, and second amplitude envelope adjustment parameters to meet user needs.

[0098] Optionally, the noise type can be white noise, pink noise, brown noise, etc., and the filter cutoff frequency can be in the range of 20~20000Hz. Of course, it can also be any other reasonable value, which is not limited in the embodiments of this disclosure.

[0099] Optionally, if the noise audio feature parameters also include a second note velocity parameter, the method for generating the noise signal of the target instrument can be as follows: Based on formula (12), the fundamental tone signal of the target instrument is generated: Formula (12) According to the above formula (12), the user can adjust the fundamental tone signal of the generated target instrument by inputting the noise type, filter cutoff frequency, filter type, second amplitude envelope adjustment parameter and second note velocity parameter, so as to meet the user's needs.

[0100] It is understood that the above are only examples of several possible noise functions, and the noise function can also be any other reasonable function. This disclosure does not specifically limit the embodiments in this regard.

[0101] In another alternative approach, at least one noise audio feature parameter can be input into the noise generation model to obtain the output noise signal of the target instrument.

[0102] The noise model can be any pre-trained machine learning model. After obtaining at least one noise audio feature parameter, it can be input into the noise generation model to obtain the output noise signal.

[0103] Step S104: Based on the target synthesis control parameters, the gene signal and noise signal are synthesized to obtain the instrument audio signal that matches the target timbre.

[0104] The fundamental tone signal and noise signal of the target instrument are synthesized and processed to make its timbre closer to the real drum sound audio.

[0105] In one alternative approach, the synthesis processing method can be: weighting the pitch signal and noise signal based on the target synthesis control parameters to obtain the instrument audio signal.

[0106] After obtaining the target synthesis control parameters input by the user or the default parameters, the pitch signal and noise audio of the target instrument can be weighted based on these parameters to obtain the synthesized instrument audio signal.

[0107] The corresponding formula can be expressed as follows: Formula (13) in, For sampling point index, For musical instrument audio signals, To synthesize control parameters for the target, The fundamental tone signal of the target instrument. This is noise audio.

[0108] It can be any value from 0 to 1, for example, A value of 0.3 can generate a snare drum tone that is primarily noise.

[0109] In the embodiments of this disclosure, in addition to the methods described above, the synthesis processing method can also be any other reasonable method, such as modulation synthesis method, subtraction synthesis method, particle synthesis method, etc.

[0110] Step S105: Generate the target arrangement audio based on the instrument audio signal.

[0111] After generating the instrument audio signal, a corresponding target arrangement audio can be generated based on the instrument audio signal. The target arrangement audio is an arrangement audio that is mixed with instrument audio signals and generated by the user according to the main melody according to their own needs. In addition to the instrument audio signal of the target instrument, the target arrangement audio can also be mixed with the audio signals of other instruments. It can be an audio segment of a complete arrangement audio or a complete arrangement audio extended from the instrument audio signal. This disclosure embodiment does not limit it.

[0112] There are several methods for generating target arrangement audio based on instrument audio signals, as follows: After generating the instrument audio signal, it can be directly identified as the target arrangement audio.

[0113] Alternatively, if the user has already input the initial arrangement audio, the processing method can be as follows: Obtain the initial arrangement audio. Combine the initial arrangement audio with the instrument audio signals to obtain the target arrangement audio.

[0114] First, the user-inputted initial arrangement audio can be obtained, which is a pre-completed arrangement audio. Then, according to the user's selection, the generated instrument audio signal can be added to the target time point of the initial arrangement audio, that is, the first audio frame of the instrument audio signal is combined with the audio frame at the target time point of the initial arrangement audio to obtain the target arrangement audio.

[0115] Optionally, the process of generating the target arranged audio may further include: obtaining distortion intensity parameters. Based on the distortion intensity parameters, the instrument audio signal is subjected to distortion processing to obtain the target arranged audio.

[0116] After obtaining the user-inputted or default distortion intensity parameters, the instrument audio signal can be weighted based on these distortion intensity parameters to obtain the synthesized instrument audio signal.

[0117] The corresponding formula can be expressed as follows: Formula (14) in, For musical instrument audio signals, Arrange audio for the target audience. For example, distortion intensity parameters, A value of 5 can generate distorted drum beats with a strong electronic feel.

[0118] Optionally, the process of generating the target arranged audio may further include: acquiring sound image parameters; generating left channel instrument audio and right channel instrument audio based on the sound image parameters and instrument audio signals; and generating the target arranged audio based on the left channel instrument audio and right channel instrument audio.

[0119] After obtaining the user-inputted or default audio parameters, the system can generate left-channel and right-channel instrument audio based on these parameters and the instrument audio signal, thereby providing users with a better listening experience.

[0120] The corresponding formula can be expressed as follows: Formula (15) in, For musical instrument audio signals, This refers to the instrument audio signal after distortion processing based on distortion intensity parameters. These are panning parameters, which can adjust the global volume. For example... When the value is 0, the sound is output only in the left channel.

[0121] Optionally, the process of generating the target arranged audio may further include: obtaining global volume parameters; adjusting the global volume of the instrument audio signal based on the global volume parameters to obtain the adjusted instrument audio signal; and generating the target arranged audio based on the adjusted instrument audio signal.

[0122] After obtaining the user-inputted or default global volume parameters, the global volume of the instrument audio signal can be adjusted based on these global volume parameters, thereby providing the user with a better listening experience.

[0123] In addition, to better understand the solutions of the embodiments of this application, the following are also provided. Figure 2 The process shown is as follows: Figure 2 For detailed explanations of the process shown, please refer to the previous description, which will not be repeated here.

[0124] All of the above-mentioned optional technical solutions can be combined in any way to form optional embodiments of this disclosure, and will not be described in detail here.

[0125] The solution mentioned in this embodiment can adjust the instrument audio characteristic parameters and noise audio characteristic parameters according to different scenario requirements of the target arranged audio, thereby realizing a variety of dynamically adjustable target arranged audio and effectively improving the arrangement effect.

[0126] This disclosure provides an apparatus for generating arranged audio, which may be the computer device described in the above embodiments, such as... Figure 3 As shown, the device includes: The acquisition module 310 is used to acquire at least one musical instrument audio feature parameter and at least one noise audio feature parameter; The pitch generation module 320 is used to generate the pitch signal of the target instrument based on the at least one instrument audio feature parameter; The noise generation module 330 is used to generate a noise signal of the target musical instrument based on the at least one noise audio feature parameter. The synthesis module 340 is used to synthesize the gene signal and the noise signal based on the target synthesis control parameters to obtain the instrument audio signal that matches the target timbre of the target instrument. The arrangement generation module 350 is used to generate target arrangement audio based on the instrument audio signal.

[0127] In an alternative embodiment, the pitch generation module 320 is configured to: The at least one instrument audio characteristic parameter is input into the oscillator function to generate the fundamental tone signal of the target instrument.

[0128] In one alternative approach, the oscillator function includes a waveform function and a first amplitude envelope function, and the instrument audio characteristic parameters include waveform adjustment parameters and first amplitude envelope adjustment parameters; The pitch generation module 320 is used for: The waveform adjustment parameters are input into the waveform function to generate a waveform signal; Input the first amplitude envelope adjustment parameter into the first amplitude envelope function to obtain the adjusted first amplitude envelope function; Based on the adjusted first amplitude envelope function and the waveform signal, the fundamental tone signal of the target musical instrument is generated.

[0129] In one alternative approach, the instrument audio characteristic parameters further include a first note velocity parameter; The pitch generation module 320 is used for: Based on the adjusted first amplitude envelope function, the waveform signal is subjected to amplitude envelope modulation to obtain the modulated waveform signal. Based on the first note velocity parameter, the modulated waveform signal is subjected to overall amplitude modulation to obtain the fundamental tone signal of the target instrument.

[0130] In one alternative approach, the waveform function includes a pitch envelope function and a low-frequency oscillation function, and the waveform adjustment parameters include a fundamental frequency parameter, a pitch envelope adjustment parameter, and an oscillation adjustment parameter. The pitch generation module 320 is used for: The pitch envelope adjustment parameters are input into the pitch envelope function to obtain the adjusted pitch envelope function; The oscillation adjustment parameters are input into the low-frequency oscillation function to obtain the adjusted low-frequency oscillation function; The waveform signal is generated based on the fundamental frequency parameters, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function.

[0131] In an alternative embodiment, the pitch generation module 320 is configured to: Based on formula (1), the fundamental tone signal of the target musical instrument is generated: Formula (1) in, For sampling point index, The fundamental tone signal, The adjusted first amplitude envelope function, The waveform signal, The fundamental frequency parameter, The adjusted pitch envelope function, This is the adjusted low-frequency oscillation function.

[0132] In an alternative embodiment, the noise generation module 330 is configured to: The at least one noise audio feature parameter is input into a noise function to generate the fundamental tone signal of the target musical instrument.

[0133] In one alternative approach, the noise function includes a filter and a second amplitude envelope function, and the noise audio feature parameters include filter parameters and a second amplitude envelope adjustment parameter. The noise generation module 330 is used for: The filter parameters are input into the filter to generate an initial noise signal; Input the second amplitude envelope adjustment parameter into the second amplitude envelope function to obtain the adjusted second amplitude envelope function; Based on the adjusted second amplitude envelope function and the initial noise signal, the fundamental tone signal of the target musical instrument is generated.

[0134] In one alternative approach, the noise audio characteristic parameters further include a second note velocity parameter; The noise generation module 330 is used for: Based on the adjusted second amplitude envelope function, the initial noise signal is subjected to amplitude envelope modulation to obtain the modulated initial noise signal; Based on the second note velocity parameter, the modulated initial noise signal is subjected to overall amplitude modulation to obtain the fundamental tone signal of the target instrument.

[0135] In one alternative approach, the filter parameters include at least one of noise type, filter cutoff frequency, and filter type.

[0136] In an alternative embodiment, the noise generation module 330 is configured to: Based on formula (2), the fundamental tone signal of the target musical instrument is generated: Formula (2) in, The noise signal, The adjusted second amplitude envelope function, For the filter, For the aforementioned noise type, The cutoff frequency of the filter is... This refers to the filter type.

[0137] In an alternative embodiment, the synthesis module 340 is configured to: Based on the target synthesis control parameters, the pitch signal and the noise signal are weighted and processed to obtain the instrument audio signal.

[0138] In an alternative embodiment, the acquisition module 310 is further configured to: Obtain the distortion intensity parameters; The music arrangement generation module 350 is used for: Based on the distortion intensity parameter, the instrument audio signal is distorted to obtain the target arranged audio.

[0139] In an alternative embodiment, the acquisition module 310 is further configured to: Obtain the initial arrangement audio; The music arrangement generation module 350 is used for: The initial arrangement audio and the instrument audio signal are synthesized to obtain the target arrangement audio.

[0140] In an alternative embodiment, the acquisition module 310 is further configured to: Obtain audio-visual parameters; The music arrangement generation module 350 is used for: Based on the sound image parameters and the instrument audio signal, generate left channel instrument audio and right channel instrument audio; The target arrangement audio is generated based on the left channel instrument audio and the right channel instrument audio.

[0141] In an alternative embodiment, the acquisition module 310 is further configured to: Get global volume parameters; The music arrangement generation module 350 is used for: Based on the global volume parameters, the global volume of the instrument audio signal is adjusted to obtain the adjusted instrument audio signal; The target arrangement audio is generated based on the adjusted instrument audio signal.

[0142] It should be noted that the above-described audio generation device is only illustrated by the division of the functional modules described above. In practical applications, the functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. Furthermore, the audio generation device and the audio generation method described above belong to the same concept, and their specific implementation process can be found in the method embodiment, which will not be repeated here.

[0143] Figure 4A structural block diagram of a terminal 400 provided in an exemplary embodiment of this disclosure is shown. The terminal can be a computer device as described in the above embodiments. The terminal 400 can be a smartphone, tablet computer, MP3 player (Moving Picture Experts Group Audio Layer III), MP4 player (Moving Picture Experts Group Audio Layer IV), laptop computer, or desktop computer. The terminal 400 may also be referred to as a user device, portable terminal, laptop terminal, desktop terminal, or other names.

[0144] Typically, terminal 400 includes a processor 401 and a memory 402.

[0145] Processor 401 may include one or more processing cores, such as a quad-core processor, an octa-core processor, etc. Processor 401 may be implemented using at least one hardware form selected from DSP (digital signal processing), FPGA (field-programmable gate array), and PLA (programmable logic array). Processor 401 may also include a main processor and a coprocessor. The main processor, also known as a CPU (central processing unit), is used to process data in the wake-up state; the coprocessor is a low-power processor used to process data in the standby state. In some embodiments, processor 401 may integrate a GPU (graphics processing unit), which is responsible for rendering and drawing the content to be displayed on the screen. In some embodiments, processor 401 may also include an AI (artificial intelligence) processor, which is used to handle computational operations related to machine learning.

[0146] The memory 402 may include one or more computer-readable storage media, which may be non-transitory. The memory 402 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices or flash memory devices. In some embodiments, the non-transitory computer-readable storage media in the memory 402 are used to store at least one instruction, which is executed by the processor 401 to implement the method for generating arranged audio provided in the method embodiments of this disclosure.

[0147] In some embodiments, the terminal 400 may also optionally include a peripheral device interface 403 and at least one peripheral device. The processor 401, memory 402, and peripheral device interface 403 can be connected via a bus or signal line. Each peripheral device can be connected to the peripheral device interface 403 via a bus, signal line, or circuit board. Specifically, the peripheral device includes at least one of the following: a radio frequency circuit 404, a display screen 405, a camera 406, an audio circuit 407, a positioning component 408, and a power supply 409.

[0148] Peripheral device interface 403 can be used to connect at least one I / O (input / output) related peripheral device to processor 401 and memory 402. In some embodiments, processor 401, memory 402 and peripheral device interface 403 are integrated on the same chip or circuit board; in some other embodiments, any one or two of processor 401, memory 402 and peripheral device interface 403 can be implemented on separate chips or circuit boards, which is not limited in this embodiment.

[0149] The radio frequency (RF) circuit 404 is used to receive and transmit RF (radio frequency) signals, also known as electromagnetic signals. The RF circuit 404 communicates with communication networks and other communication devices via electromagnetic signals. The RF circuit 404 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals back into electrical signals. Optionally, the RF circuit 404 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, etc. The RF circuit 404 can communicate with other terminals through at least one wireless communication protocol. This wireless communication protocol includes, but is not limited to: metropolitan area networks (MANs), various generations of mobile communication networks (2G, 3G, 4G, and 5G), wireless local area networks (WLANs), and / or WiFi (wireless fidelity) networks. In some embodiments, the RF circuit 404 may also include circuitry related to NFC (near field communication), which is not limited in this disclosure.

[0150] Display screen 405 is used to display a user interface (UI). This UI may include graphics, text, icons, videos, and any combination thereof. When display screen 405 is a touch display screen, it also has the ability to collect touch signals on or above its surface. These touch signals can be input as control signals to processor 401 for processing. In this case, display screen 405 can also be used to provide virtual buttons and / or a virtual keyboard, also known as soft buttons and / or a soft keyboard. In some embodiments, there may be one display screen 405, which serves as the front panel of terminal 400; in other embodiments, there may be at least two display screens 405, respectively disposed on different surfaces of terminal 400 or in a folded design; in still other embodiments, display screen 405 may be a flexible display screen, disposed on a curved or folded surface of terminal 400. Furthermore, display screen 405 may be configured as a non-rectangular irregular shape, i.e., a non-rectangular screen. Display screen 405 may be made of materials such as LCD (liquid crystal display) or OLED (organic light-emitting diode).

[0151] The camera assembly 406 is used to acquire images or videos. Optionally, the camera assembly 406 includes a front-facing camera and a rear-facing camera. Typically, the front-facing camera is located on the front panel of the terminal, and the rear-facing camera is located on the back of the terminal. In some embodiments, there are at least two rear-facing cameras, which are any one of a main camera, a depth-sensing camera, a wide-angle camera, and a telephoto camera, to achieve background blurring by fusion of the main camera and the depth-sensing camera, panoramic shooting by fusion of the main camera and the wide-angle camera, VR (virtual reality) shooting, or other fusion shooting functions. In some embodiments, the camera assembly 406 may also include a flash. The flash can be a single-color temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm-light flash and a cool-light flash, which can be used for light compensation at different color temperatures.

[0152] The audio circuit 407 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, converting them into electrical signals that are input to the processor 401 for processing, or to the radio frequency circuit 404 for voice communication. For stereo sound acquisition or noise reduction purposes, multiple microphones may be used, each positioned at a different location on the terminal 400. The microphone may also be an array microphone or an omnidirectional microphone. The speaker is used to convert electrical signals from the processor 401 or the radio frequency circuit 404 into sound waves. The speaker may be a conventional diaphragm speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can convert electrical signals not only into audible sound waves but also into inaudible sound waves for purposes such as distance measurement. In some embodiments, the audio circuit 407 may also include a headphone jack.

[0153] The positioning component 408 is used to determine the current geographic location of the terminal 400 in order to enable navigation or LBS (location-based service). The positioning component 408 can be a positioning component based on GPS (global positioning system), BeiDou system, Grenadin system, or Galileo system.

[0154] Power supply 409 is used to power the various components in terminal 400. Power supply 409 can be AC ​​power, DC power, a disposable battery, or a rechargeable battery. When power supply 409 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0155] In some embodiments, the terminal 400 further includes one or more sensors 410. The one or more sensors 410 include, but are not limited to: an accelerometer 411, a gyroscope 412, a pressure sensor 413, a fingerprint sensor 414, an optical sensor 415, and a proximity sensor 416.

[0156] Accelerometer 411 can detect the magnitude of acceleration along the three coordinate axes of a coordinate system established by terminal 400. For example, accelerometer 411 can be used to detect the components of gravitational acceleration along the three coordinate axes. Processor 401 can control display screen 405 to display the user interface in either a landscape or portrait view based on the gravitational acceleration signal acquired by accelerometer 411. Accelerometer 411 can also be used for games or for acquiring user motion data.

[0157] The gyroscope sensor 412 can detect the orientation and rotation angle of the terminal 400. The gyroscope sensor 412, in conjunction with the accelerometer sensor 411, can collect 3D motion data from the user on the terminal 400. Based on the data collected by the gyroscope sensor 412, the processor 401 can perform the following functions: motion sensing (e.g., changing the UI based on the user's tilt), image stabilization during shooting, game control, and inertial navigation.

[0158] The pressure sensor 413 can be disposed on the side bezel of the terminal 400 and / or on the lower layer of the display screen 405. When the pressure sensor 413 is disposed on the side bezel of the terminal 400, it can detect the user's grip signal on the terminal 400, and the processor 401 can perform left / right hand recognition or quick operation based on the grip signal collected by the pressure sensor 413. When the pressure sensor 413 is disposed on the lower layer of the display screen 405, the processor 401 can control the operable controls on the UI interface based on the user's pressure operation on the display screen 405. The operable controls include at least one of button controls, scroll bar controls, icon controls, and menu controls.

[0159] The fingerprint sensor 414 is used to collect the user's fingerprint. The processor 401 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 414, or the fingerprint sensor 414 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as trusted, the processor 401 authorizes the user to perform relevant sensitive operations, including unlocking the screen, viewing encrypted information, downloading software, making payments, and changing settings. The fingerprint sensor 414 can be located on the front, back, or side of the terminal 400. When the terminal 400 has physical buttons or a manufacturer's logo, the fingerprint sensor 414 can be integrated with the physical buttons or manufacturer's logo.

[0160] An optical sensor 415 is used to collect ambient light intensity. In one embodiment, the processor 401 can control the display brightness of the display screen 405 based on the ambient light intensity collected by the optical sensor 415. Specifically, when the ambient light intensity is high, the display brightness of the display screen 405 is increased; when the ambient light intensity is low, the display brightness of the display screen 405 is decreased. In another embodiment, the processor 401 can also dynamically adjust the shooting parameters of the camera assembly 406 based on the ambient light intensity collected by the optical sensor 415.

[0161] The proximity sensor 416, also known as a distance sensor, is typically located on the front panel of the terminal 400. The proximity sensor 416 is used to detect the distance between the user and the front of the terminal 400. In one embodiment, when the proximity sensor 416 detects that the distance between the user and the front of the terminal 400 is gradually decreasing, the processor 401 controls the display screen 405 to switch from a screen-on state to a screen-off state; when the proximity sensor 416 detects that the distance between the user and the front of the terminal 400 is gradually increasing, the processor 401 controls the display screen 405 to switch from a screen-off state to a screen-on state.

[0162] Those skilled in the art will understand that Figure 4 The structure shown does not constitute a limitation on terminal 400 and may include more or fewer components than shown, or combine certain components, or use different component arrangements.

[0163] Figure 5 This is a schematic diagram of a server structure provided in an embodiment of this disclosure. The server 500 can vary significantly due to different configurations or performance. It may include one or more central processing units (CPUs) 501 and one or more memories 502. The memories 502 store at least one instruction, which is loaded and executed by the processors 501 to implement the methods provided in the various method embodiments described above. Of course, the server may also have wired or wireless network interfaces, a keyboard, and input / output interfaces for input and output. The server may also include other components for implementing device functions, which will not be elaborated upon here.

[0164] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including instructions that can be executed by a processor in a terminal to complete the method for generating arranged audio in the above embodiments. This computer-readable storage medium can be non-transitory. For example, the computer-readable storage medium can be ROM (read-only memory), RAM (random access memory), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0165] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0166] It should be noted that all information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between user terminals and other devices) involved in this disclosure are authorized by the user or fully authorized by all parties, and the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions. For example, the "musical instrument audio characteristic parameters and noise audio characteristic parameters," "oscillator function," "noise function," etc. involved in this disclosure were all obtained with full authorization.

[0167] The above description is merely an optional embodiment of this disclosure and is not intended to limit this disclosure. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this disclosure should be included within the protection scope of this disclosure.

Claims

1. A method for generating arranged audio, characterized in that, The method includes: Obtain at least one instrument audio feature parameter and at least one noise audio feature parameter; Based on the at least one musical instrument audio feature parameter, generate the fundamental tone signal of the target musical instrument; Based on the at least one noise audio feature parameter, generate the noise signal of the target musical instrument; Based on the target synthesis control parameters, the gene signal and the noise signal are synthesized to obtain the instrument audio signal that matches the target timbre of the target instrument; Based on the instrument audio signal, the target arrangement audio is generated.

2. The method according to claim 1, characterized in that, The step of generating the fundamental tone signal of the target instrument based on the at least one instrument audio feature parameter includes: The at least one instrument audio characteristic parameter is input into the oscillator function to generate the fundamental tone signal of the target instrument.

3. The method according to claim 2, characterized in that, The oscillator function includes a waveform function and a first amplitude envelope function, and the instrument audio characteristic parameters include waveform adjustment parameters and first amplitude envelope adjustment parameters; The step of inputting the at least one instrument audio feature parameter into an oscillator function to obtain the fundamental tone signal of the target instrument includes: The waveform adjustment parameters are input into the waveform function to generate a waveform signal; Input the first amplitude envelope adjustment parameter into the first amplitude envelope function to obtain the adjusted first amplitude envelope function; Based on the adjusted first amplitude envelope function and the waveform signal, the fundamental tone signal of the target musical instrument is generated.

4. The method according to claim 3, characterized in that, The instrument audio characteristic parameters also include the first note velocity parameter; The step of generating the fundamental tone signal of the target musical instrument based on the adjusted first amplitude envelope function and the waveform signal includes: Based on the adjusted first amplitude envelope function, the waveform signal is subjected to amplitude envelope modulation to obtain the modulated waveform signal. Based on the first note velocity parameter, the modulated waveform signal is subjected to overall amplitude modulation to obtain the fundamental tone signal of the target instrument.

5. The method according to claim 3, characterized in that, The waveform function includes a pitch envelope function and a low-frequency oscillation function, and the waveform adjustment parameters include a fundamental frequency parameter, a pitch envelope adjustment parameter, and an oscillation adjustment parameter. The step of inputting the waveform adjustment parameters into the waveform function to generate a waveform signal includes: The pitch envelope adjustment parameters are input into the pitch envelope function to obtain the adjusted pitch envelope function; The oscillation adjustment parameters are input into the low-frequency oscillation function to obtain the adjusted low-frequency oscillation function; The waveform signal is generated based on the fundamental frequency parameters, the adjusted pitch envelope function, and the adjusted low-frequency oscillation function.

6. The method according to claim 5, characterized in that, The step of generating the fundamental tone signal of the target musical instrument based on the adjusted first amplitude envelope function and the waveform signal includes: Based on formula (1), the fundamental tone signal of the target musical instrument is generated: Official (1) in, For sampling point index, The fundamental tone signal, The adjusted first amplitude envelope function, The waveform signal, The fundamental frequency parameter, The adjusted pitch envelope function, This is the adjusted low-frequency oscillation function.

7. The method according to claim 1, characterized in that, The step of generating the noise signal of the target musical instrument based on the at least one noise audio feature parameter includes: The noise signal of the target musical instrument is generated by inputting the at least one noise audio feature parameter into the noise function.

8. The method according to claim 7, characterized in that, The noise function includes a filter and a second amplitude envelope function, and the noise audio feature parameters include filter parameters and a second amplitude envelope adjustment parameter; The step of inputting the at least one noise audio feature parameter into a noise function to generate a noise signal for the target instrument includes: The filter parameters are input into the filter to generate an initial noise signal; Input the second amplitude envelope adjustment parameter into the second amplitude envelope function to obtain the adjusted second amplitude envelope function; Based on the adjusted second amplitude envelope function and the initial noise signal, the noise signal of the target musical instrument is generated.

9. The method according to claim 8, characterized in that, The noise audio characteristic parameters also include the second note velocity parameter; The step of generating the noise signal for the target musical instrument based on the adjusted second amplitude envelope function and the initial noise signal includes: Based on the adjusted second amplitude envelope function, the initial noise signal is subjected to amplitude envelope modulation to obtain the modulated initial noise signal; Based on the second note velocity parameter, the modulated initial noise signal is subjected to overall amplitude modulation to obtain the noise signal of the target instrument.

10. The method according to claim 8, characterized in that, The filter parameters include at least one of noise type, filter cutoff frequency, and filter type.

11. The method according to claim 10, characterized in that, The step of generating the noise signal for the target musical instrument based on the adjusted second amplitude envelope function and the initial noise signal includes: Based on formula (2), the noise signal of the target musical instrument is generated: Official (2) in, The noise signal, The adjusted second amplitude envelope function, For the filter, For the aforementioned noise type, The cutoff frequency of the filter is... This refers to the filter type.

12. The method according to claim 1, characterized in that, The process of synthesizing the gene signal and the noise signal based on the target synthesis control parameters to obtain the instrument audio signal that matches the target timbre of the target instrument includes: Based on the target synthesis control parameters, the pitch signal and the noise signal are weighted and processed to obtain the instrument audio signal.

13. The method according to claim 1, characterized in that, The method further includes: Obtain the distortion intensity parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the distortion intensity parameter, the instrument audio signal is distorted to obtain the target arranged audio.

14. The method according to claim 1, characterized in that, The method further includes: Obtain the initial arrangement audio; The step of generating the target arrangement audio based on the instrument audio signal includes: The initial arrangement audio and the instrument audio signal are synthesized to obtain the target arrangement audio.

15. The method according to claim 1, characterized in that, The method further includes: Obtain audio-visual parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the sound image parameters and the instrument audio signal, generate left channel instrument audio and right channel instrument audio; The target arrangement audio is generated based on the left channel instrument audio and the right channel instrument audio.

16. The method according to claim 1, characterized in that, The method further includes: Get global volume parameters; The step of generating the target arrangement audio based on the instrument audio signal includes: Based on the global volume parameters, the global volume of the instrument audio signal is adjusted to obtain the adjusted instrument audio signal; The target arrangement audio is generated based on the adjusted instrument audio signal.

17. A computer device, characterized in that, The computer device includes a processor and a memory, the memory storing at least one instruction, which is loaded and executed by the processor to perform the operations performed by the method for generating arranged audio as described in any one of claims 1 to 16.

18. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, which is loaded and executed by a processor to perform the operations performed by the method for generating arranged audio as described in any one of claims 1 to 16.

19. A computer program product, characterized in that, The computer program product includes at least one instruction, which is loaded and executed by a processor to perform the operation of the method for generating arranged audio as described in any one of claims 1 to 16.

Images