Audio playing method, system and device and computer program product

Abstract

The invention provides an audio playing method, device and system and a computer program product. The method comprises the following steps: acquiring a first input input through a user interface, wherein the first input comprises music parameter information; obtaining a second input input through the user interface, wherein the second input comprises sound effect data information; determining a target music clip based on the first input, wherein the target music clip is described by a target music parameter; determining a target sound effect data set based on the second input, wherein the target sound effect data set comprises audio samples of different drum sound parts with the same sound effect style; and driving the target sound effect data set to play based on the target music clip. By selecting different music parameters and sound effect styles by a user, real-time adjustment of playing parameters and playing contents can be realized, the playing contents can be enriched, and storage resource space can be saved.

Description

Technical Field

[0001] The present invention relates to the field of virtual musical instrument technology, and more particularly to an audio playback method, system, device and computer program product for use in games. Background Art

[0002] In recent years, more and more games have introduced musical instrument performance and ensemble into various types of games as a leisure way to enrich the game experience. For example, the smart drummer is an automatic accompaniment system for the drum part of a musical instrument ensemble, which can greatly optimize the overall experience of musical instrument performance and ensemble in the game. The smart drummer currently used in games has a single function, cannot achieve real-time adjustment of music parameters, and the performance content is too simple. If you need to be able to automatically play a drum part that can be customized to a certain extent through very simple operations, and some music parameters and content such as the performance speed, style, and timbre of this drum part must be controllable and variable to fit various performance styles, it is often difficult and costly for game developers to develop such a system through traditional methods. This is also the main reason why smart drummers have never appeared in games.

[0003] The present application provides an audio playback system that can achieve real-time adjustment of music parameters and rich performance and selection in the game, while also taking into account its sound quality, interactive friendliness and reasonable development workload. Summary of the invention

[0004] One or more embodiments of the present specification provide an audio playback method, the method comprising obtaining a first input input through a user interface, the first input comprising music parameter information; obtaining a second input input through the user interface, the second input comprising sound effect data information; determining a target music clip based on the first input, the target music clip being described by target music parameters; determining a target sound effect data set based on the second input, the target sound effect data set comprising audio samples of different drum parts having the same sound effect style; and driving the target sound effect data set to play based on the target music clip.

[0005] According to the audio playback method provided by one or more embodiments of the present specification, the music parameter information includes performance style information, performance density information and performance intensity information, and determining the target music clip based on the first input includes determining the target performance style, target performance density and target performance intensity based on the performance style information, performance density information and performance intensity information; and determining the target music clip from multiple preset music clips based on the target performance style, target performance density and target performance intensity, each preset music clip in the multiple preset music clips is described by multiple music parameters, and the preset music clips are stored in a digital music interface (MIDI) file format.

[0006] According to the audio playing method provided by one or more embodiments of the present specification, the same timbre in multiple preset music clips is at the same pitch.

[0007] According to the audio playback method provided by one or more embodiments of the present specification, the sound effect data information includes sound effect style information, and determining the target sound effect data set based on the second input includes determining the target sound effect data set from multiple preset sound effect data sets based on the sound effect style information, each preset sound effect data set in the multiple preset sound effect data sets includes audio samples of each drum part in a group of drum parts, and the audio samples of each drum part correspond to the same sound effect style.

[0008] The audio playback method provided according to one or more embodiments of the present specification also includes: obtaining a third input input through a user interface, the third input including a target beat rate (BPM) set by a user; determining a target music clip playback speed based on a mapping relationship between the beat rate and the music clip playback speed and the target beat rate; and playing a target sound effect data set based on the target music clip playback speed.

[0009] According to the audio playback method provided by one or more embodiments of the present specification, the user interface includes a style selector and an X / Y controller, and obtaining a first input input by a user through the user interface includes: obtaining performance style information in the first input through the style selector; and obtaining performance density information and performance intensity information in the first input through the X / Y controller.

[0010] According to the audio playback method provided by one or more embodiments of the present specification, the user interface includes a sound effect selector, and obtaining a second input input through the user interface includes: obtaining sound effect data information in the second input through the sound effect selector.

[0011] One or more embodiments of the present specification provide an audio playback device, including a processor and a memory, wherein the memory stores a computer program or a computer executable instruction, and when the computer program or the computer executable instruction is executed by the processor, the audio playback method provided by one or more embodiments of the present specification is implemented.

[0012] One or more embodiments of the present specification provide a computer program product, characterized in that it includes a computer program, and when at least a part of the computer program is executed by a processor, it can implement the audio playback method provided by one or more embodiments of the specification.

[0013] One or more embodiments of the present specification provide an audio playback system, including: a game engine, including a user interface, the user interface including a music selection module and a sound effect selection module, the music selection module is used to obtain a first input, the first input includes music parameter information, and the sound effect selection module is used to obtain a second input, the second input includes sound effect data information. The system also includes an audio engine, the audio engine includes: a music module and a sound effect module, the music module is configured to determine a target music clip based on the first input, the target music clip is described by a target music parameter; the sound effect module is configured to determine a target sound effect data set based on the second input, the target sound effect data set includes audio samples of different drum parts with the same sound effect style; and the target sound effect data set is driven to play based on the target music clip.

[0014] According to the audio playback system provided by one or more embodiments of the present specification, the music selection module includes a style selector and an X / Y controller, the style selector is used to input the performance style information in the music parameter information, and the X / Y controller is used to input the performance density information and performance intensity information in the music parameter information.

[0015] According to the audio playback system provided by one or more embodiments of the present specification, the user interface also includes a playback control module, the playback control module includes a beat number (BPM) selector, which is used to obtain a third input, and the third input includes a target beat number set by a user. The audio engine is further configured to: determine the target music piece playback speed based on the mapping relationship between the beat number and the music piece playback speed and the target beat number; and play the target sound effect data set based on the target music piece playback speed.

[0016] The audio playback system provided according to one or more embodiments of the present specification further includes a storage module for storing multiple preset sound effect data sets, and the sound effect module is further configured to: determine a target sound effect data set from multiple preset sound effect data sets based on sound effect data information, each of the multiple preset sound effect data sets includes audio samples of each drum part in a group of drum parts, and the audio samples of each drum part correspond to the same sound effect style.

[0017] The audio playback system provided according to one or more embodiments of the present specification further includes a storage module for storing multiple preset music clips, and the music module is further configured to: determine a target music clip from the multiple preset music clips based on a first input, each of the multiple preset music clips is described by multiple music parameters, and the preset music clips are stored in a digital music interface (MIDI) file format. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] This specification will be further explained by way of exemplary embodiments, which will be described in detail by way of the accompanying drawings. The same numbers in the drawings represent the same structures or steps.

[0019] Figure 1 It is a schematic diagram of an audio playback system according to some embodiments of the present specification.

[0020] Figure 2 It is a specific embodiment of the audio playback system shown in some embodiments of this specification.

[0021] Figure 3 It is a schematic diagram of the music module configuration shown in some embodiments of this specification.

[0022] Figure 4 It is a schematic diagram of the configuration of the sound effect module according to some embodiments of this specification.

[0023] Figure 5 It is a schematic diagram of controlling BPM based on RTPC technology according to some embodiments of this specification.

[0024] Figure 6 It is a schematic diagram of selecting or switching the performance density based on RTPC according to some embodiments of the present application.

[0025] Figure 7 It is a schematic diagram of a user interface according to some embodiments of the present application.

[0026] Figure 8 is a schematic diagram of an X / Y controller according to some embodiments of the present application.

[0027] Fig. 9 is a schematic diagram of a BPM selector according to some embodiments of the present application.

[0028] Fig.10 A flowchart of an audio playback method according to some embodiments of this specification.

[0029] Fig.11 Schematic diagram of an audio playback device according to some embodiments of this specification.

[0030] Fig.12 An electronic device for audio playback as shown in some embodiments of this specification. DETAILED DESCRIPTION

[0031] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be described in detail below with reference to the accompanying drawings. Obviously, the contents described below are some examples or embodiments of this specification. For ordinary technicians in this field, without paying creative work, the technical solutions or means disclosed in this specification can also be applied to other scenarios based on these technical contents.

[0032] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.

[0033] Unless otherwise specified, technical terms used in this specification to describe components, elements, etc. do not refer to the singular, but may also include the plural. Generally speaking, terms such as "include", "comprise", etc. only indicate that the steps, elements, or components that have been clearly identified are included, and these steps, elements, and components do not constitute an exclusive list, such as the method or device described may also include other steps or components.

[0034] Flowcharts are used in this specification to illustrate the operation steps performed by the device or system of the relevant embodiments, but unless otherwise specified, the order used to describe these steps should not be understood as a limitation on the order in which the steps are performed. A person of ordinary skill in the art can adjust the order in which these steps are performed based on the knowledge and information conveyed by the embodiments of this specification, and the adjustment includes but is not limited to swapping the order, merging multiple steps, and splitting a certain step.

[0035] The triggering of musical instrument sounds including drums in games usually does not include the "real-time triggering of a single timbre" mode or the "looping of a certain musical instrument audio" mode. The "real-time triggering of a single timbre" mode means that a sound is played every time the user clicks a button. The "looping of a certain musical instrument audio" mode means that a pre-defined audio clip is played every time the user clicks a button. For example, in the "real-time triggering of a single timbre", when the bass drum button and the snare drum button of the drum set are clicked, "dong" and "da" are heard respectively. In the "looping of a certain musical instrument audio" mode, when the user clicks the start button, the pre-prepared audio clip "dong~dadong~dongda~" starts to play in a loop until the user clicks the stop button. The user can trigger the sounds of other musical instruments at the same time during the looping of the drum part to achieve "ensemble" in the game. When playing drums with rich expressiveness (especially complex and fast drum sets) through "real-time triggering of a single timbre", the user's playing ability is higher, because the user needs to play each part through the touch screen or computer keyboard like a professional drummer. For this reason, this problem is usually improved in games by "looping a certain section of performance audio". In some embodiments, the audio playback in the game is performed in the "looping a certain section of performance audio" mode, and the BPM may not be adjusted in real time, and the performance is single, with almost no adjustable and variable performance parameters. In some embodiments, if richer performance content is needed, it can only be done by adding more audio clips, which consumes a lot of additional storage space, thereby causing the game package to increase unnecessarily. In some embodiments, if more audio clips are added, in addition to the above-mentioned problem of occupying additional storage resources, the user's retrieval and selection process will become more complicated and inconvenient, which deviates from the original intention of "reducing difficulty" to a certain extent.

[0036] To this end, some embodiments of the present specification propose an audio playback system, which can adjust the performance parameters and performance content in real time in the game, and interact in a concise and friendly way, so as to achieve flexible control and rich performance. The audio playback system includes a game engine, including a user interface, the user interface includes a music selection module and a sound effect selection module, the music selection module is used to obtain a first input of the user, the first input includes music parameter information; the sound effect selection module is used to obtain a second input of the user, and the second input includes sound effect data information. The audio playback system also includes an audio engine. The audio engine includes a music module and a sound effect module, the music module is configured to determine a target music clip based on the first input, the target music clip includes a target music parameter; the sound effect module is configured to determine a target sound effect data set based on the second input, the target sound effect data set includes audio samples of different drum parts with the same sound effect style; and the target sound effect data set is driven to play based on the target music clip. By selecting different music parameters and sound effect styles by the user, the real-time adjustment of the performance parameters can be achieved. By presetting multiple music clips and presetting multiple sound effect data sets, and by randomly combining music clips and sound effect data sets, the performance content can be enriched and storage resource space can be saved.

[0037] Figure 1 It is a schematic diagram of a music playing system according to some embodiments of the present specification.

[0038] The audio playback system 100 can be used for the audio implementation of various virtual musical instruments (e.g., drums, piano, etc.) in the game. In some embodiments, the audio playback system can also be called a virtual musical instrument system, for example, a virtual drummer system. In this specification, the virtual drummer system is used as an example for illustration, but this specification can not be limited to the scope of the embodiments. The virtual drummer system can use computer software to simulate the sound (timbre and performance) of the real drum part. The drum part includes drums and cymbals. Drums can include military drums, bass drums, floor drums, toms, etc. Cymbals can include hi-hats, hanging cymbals, ride cymbals, stacked cymbals, Chinese cymbals, splash cymbals, etc. In some embodiments, drums can also include conga drums, bongo drums, tambourines, etc.

[0039] At least some of the components in the audio playback system 100 can be integrated into the terminal 110. In some embodiments, the terminal 110 may include a mobile device, a tablet computer, ..., a laptop computer, or any combination thereof. For example, the mobile device may include a mobile phone, a personal digital assistant (PDA), a gaming device, a navigation device, a laptop computer, a tablet computer, a desktop computer, or any combination thereof. In some embodiments, the terminal 110 may include an input device, an output device, or the like. The input device may include alphanumeric keys and other keys, which may be input via a keyboard, a touch screen (e.g., with tactile or tactile feedback), voice input, eye tracking input, a brain monitoring system, or any other similar input mechanism. The input information received by the input device may be transmitted to the processing device 112 via, for example, a bus for further processing. Other types of input devices may include a cursor control device, such as a mouse, a trackball, or a cursor direction key, etc. The output device may include a display, a speaker, or the like, or a combination thereof.

[0040] Terminal 110 may provide a user interface. A user interface refers to an interface for interaction between a user and the audio playback system 100, the processing device 112, or software. In some implementations, the user interface may include a graphical user interface, a touch user interface, a voice user interface, a natural user interface, and the like. A graphical user interface may interact using visual elements such as windows, icons, buttons, and menus. A touch user interface may allow a user to interact by touching a screen. A voice user interface may allow a user to interact with the system by voice. A natural user interface may interact using natural user behavior, such as gesture recognition, facial recognition, and the like. In some embodiments, terminal 110 may provide multiple types of user interface windows for the user to select from.

[0041] In some embodiments, the user may interact with the processing device 112 through a user interface to achieve audio playback. For example, the user may input a first input and / or a second input through the user interface, and the processing device 112 may determine the target music clip (e.g., target MIDI file) and the target sound effect data set selected by the user based on the first input and the second input, respectively. For another example, the user may input a music output parameter (e.g., target BPM) through the user interface, and the processing device 112 may adjust the playback parameter (e.g., BPM) of the target music clip (e.g., target MIDI file) according to the music output parameter input by the user. In some embodiments, the user interface may include input controls (e.g., buttons, text boxes, drop-down menus, tabs, X / Y controllers, etc.), navigation components (e.g., menu bars), information displays (e.g., labels, icons, message boxes, etc.), etc. Input controls may be used for users to input operation instructions. For example, a user may trigger the audio engine to play the target MDI file selected by the user through a play button. For another example, a user may select a performance style through a tab or a drop-down menu. For another example, a user may determine the performance density and / or performance intensity through an X / Y controller. The information display may provide prompt information to the user, such as operation prompt information for the X / Y controller. The navigation component can help users switch between different interfaces and functions. For example, the navigation component can include a game portal and an audio portal to enable switching between the game interface and the audio interface.

[0042] In some embodiments, when the audio playback system 100 is applied to a game, the user interface may be a user interface of a game engine.

[0043] The terminal 110 may include a processing device 112 and a memory (not shown).

[0044] The processing device 112 can process data obtained from the input device of the terminal 110 and / or send the processed data to the output device of the terminal 110. For example, the processing device 112 can obtain a first input from a user through a user interface, the first input including music parameter information; obtain a second input from a user through a user interface, the second input including sound effect data. The processing device 112 can determine a target music segment (e.g., a target MIDI file) based on the first input; determine a target audio data set based on the second input; and output a target audio data set based on the target music segment (e.g., a target MIDI file). The target audio data set can be output to a speaker of the terminal 110 to play the target audio sample. The target audio data set includes a plurality of target audio samples.

[0045] In some embodiments, the processing device 112 may include a processor. The processor executes computer instructions (e.g., program code) and performs the functions of the processing device 112 as described herein. For example, the computer instructions may include routines, programs, objects, components, data structures, processes, modules, and functions that perform specific functions described in this application. For example, the processor may process data obtained from a memory, terminal 110, and / or any other component. In some embodiments, the processor may include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), a high-order RISC system (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, or any combination thereof.

[0046] The processing device 112 can be used to run the audio engine 130 and / or the game engine 120. In some embodiments, the audio engine 130 and the game engine 120 can run on different processors. For example, the audio engine 130 can run on a first processor, and the game engine 120 can run on a second processor. The first processor and the second processor can communicate with each other to call and / or transmit data. For example, when a user runs the game engine 120, the user's first input and second input can be sent to the audio engine 130 through the game engine 120. The audio engine 130 can determine the target music clip and the target audio data set based on the user's first input and second input, respectively, and drive the target audio data set to play through the target music clip.

[0047] The audio engine 130 includes a music module 134 , a sound effect module 136 , and an event trigger module 132 .

[0048] The music module 134 is used to provide and switch the descriptive information of the music performance. The descriptive information of the music performance includes music parameters related to the music performance. The music parameters may include the performance content, performance intensity (performance intensity), performance density, beat, performance style, etc.

[0049] Performance density is used to describe the number of notes in a certain period of time in a music performance. The more notes are played in a certain period of time, the greater the performance density. In some implementations, performance density can be divided into different degrees. Figure 2 1 is a schematic diagram of a specific embodiment of an audio playback system according to some embodiments of this specification. Figure 2As shown, the performance density can be divided into three levels: "sparse", "medium" and "dense". For another example, the performance density can be divided into five levels: "very sparse", "sparse", "medium", "dense" and "very dense". In some embodiments, the performance density can be represented by the number of notes in a certain period of time.

[0050] Performance intensity refers to the change in volume during music performance, which can also be called performance intensity. In some implementations, performance intensity can be divided into different levels. For example, Figure 2 As shown, the playing intensity can be divided into three levels: "heavy", "medium" and "light". For another example, the playing intensity can be divided into six levels: "very weak", "weak", "medium weak", "medium strong", "strong" and "very strong". In some embodiments, the playing intensity can be represented by the size of the volume.

[0051] Beat refers to the repeated and evenly distributed time unit in music performance. The speed of the beat is determined by the tempo, and the tempo unit is beats per minute (BPM). For example, 60 BPM means there are 60 beats per minute. In some implementations, the beat can be divided into different degrees. For example, the beat can be divided into "very slow", "slow", "medium speed", "fast", "extremely fast", etc. to represent different beat speeds.

[0052] Performance styles may include classical, jazz, rock, pop, folk, neoclassical and experimental, disco, etc.

[0053] The performance content may include notes or pitches that need to be played at different time points.

[0054] Pitch refers to the high or low degree of sound. In some embodiments, pitch can be marked by octave, for example, C1, C2, C3, C4, ... B4, etc. In some implementations, pitch can be marked by twelve equal temperaments, for example, C, D, E, F, G, A, B.

[0055] In some embodiments, the music module 134 may provide and / or store multiple groups of descriptive information (i.e., music parameter information) of the music performance, corresponding to multiple preset music clips. Each preset music clip may include a time sequence of permutations and combinations of different music parameters. In some embodiments, each preset music clip may be stored in a MIDI (Musical Instrument Digital Interface) file format, also referred to as a MIDI music clip or a MIDI clip. Since MIDI files are only instructions describing music rather than actual audio, their file size is relatively small, so storage space can be saved.

[0056] A MIDI music clip is music data composed of a set of descriptive information (i.e., different music parameters) of a musical performance in time sequence. The music data instructs an electronic musical instrument (e.g., a synthesizer or computer software) how to produce or play a specific note. Furthermore, a MIDI music clip includes the distribution of different music parameters over time. For example, a MIDI music clip may include which notes of which pitch are played at different times, with what playing style, playing density, and playing strength.

[0057] In some embodiments, each MIDI file may include only one track. Since the embodiments in this specification only require the drum part, only one track is required in a MIDI file. Extra tracks are redundant and meaningless content. Setting one track can avoid unexpected loopholes. In some embodiments, when the audio playback system is used to generate sounds of multiple types of instruments, such as drums, xylophones, pianos, etc., the number of tracks in the MIDI file can be set according to the number of instrument types.

[0058] Each pitch of different pitches in the plurality of preset music clips corresponds to the timbre of a drum part (for example, a snare drum, a bass drum, a tom-tom, a hi-hat, a crash cymbal, a ride cymbal, a China cymbal, a splash cymbal, a sub-bass drum, etc.), for example, C1 corresponds to a bass drum, D1 corresponds to a snare drum, and so on.

[0059] The production of preset music clips can be designed according to the application scenario requirements. For example, multiple different preset music clips can be produced according to the performance style, performance density, and performance intensity. Each preset music clip can include a performance style, a performance density, and a performance intensity. Assuming that there are M performance styles, N performance densities, and P performance intensities, a total of M*N*P preset music clips (i.e., MIDI files) can be produced.

[0060] For example, suppose there are 2 playing styles including Rock 1 and Rock 2, 3 playing densities (sparse, medium, dense), and 3 playing strengths (heavy, medium, light), then 3*3 MIDI files are needed for each playing style, totaling 9. For example, for Rock 1, it includes MIDI file 1 (Rock 1, sparse, heavy), MIDI file 2 (Rock 1, sparse, medium), MIDI file 3 (Rock 1, sparse, light), MIDI file 4 (Rock 1, medium, heavy), MIDI file 5 (Rock 1, medium, medium), MIDI file 6 (Rock 1, medium, light), MIDI file 7 (Rock 1, dense, heavy), MIDI file 8 (Rock 1, dense, medium), and MIDI file 9 (Rock 1, dense, light). The MIDI files for Rock 2 are similar to those for Rock 1, and will not be described again here. According to the assumed 2 different playing styles, the Rock style requires a total of 2*3*3=18 MIDI clips, that is, 18 MIDI files. According to the same specifications, for each additional playing style, 9 more MIDI files are created according to the required style.

[0061] It should be noted that the more the number of splits of music parameters such as playing style, playing intensity, playing density, etc., the more preset music clips there are, that is, the more preset MIDI files there are, the finer the granularity, and the more linear the adjustment process during use. According to the same specifications, for each additional playing style, 9 more MIDI files are made according to the required playing style. Of course, the number here is just an example. The actual production can be more or less, but the content must maintain this design logic.

[0062] Each pitch in the preset music clip corresponds to a drum or cymbal with a timbre. The same timbre in different preset music clips is at the same pitch. For example, if D1 in MIDI file 1 is configured to correspond to the snare drum, then D1 in all other MIDI files is also configured to correspond to the snare drum or a timbre with a similar function to the snare drum (for example, the timbre of clapping in an electronic drum), and cannot be other timbre such as the kick drum or hi-hat. This is to facilitate the "random matching" between different music clips and different drum kit timbres. If pitch C1 in MIDI file 1 is configured to correspond to the kick drum, and the kick drum in drum kit A is also configured to correspond to pitch C1, then when this MIDI file 1 is played, the kick drum in drum kit A will be triggered when pitch C1 is played, and the sound of the kick drum will be heard. If MIDI file 2 mistakenly configures pitch C1 to correspond to the snare drum, then when MIDI file 2 is used to drive drum kit A to play, the kick drum will actually be heard, which will be inconsistent with the performance designed in MIDI file 2.

[0063] In some embodiments, the number of bars of all preset music clips is the same.

[0064] In some embodiments, the music module 134 may be constructed by a Wwise engine.

[0065] All preset music clips (e.g., preset MIDI files) can be imported into the music module. In some embodiments, these preset music clips (e.g., preset MIDI files) can be organized through a hierarchical structure according to the types of music parameters (e.g., performance density, performance intensity, and performance style) included in the music clips. The number of layers of the hierarchical structure can be related to the type of music parameters. For example, the hierarchical structure includes multiple layers, namely, the first level, the second level, and the third level. The first level is the parent level, the second level is the child of the first level, and the third level is the child of the second level.

[0066] The first, second, and third levels are all Switch containers, which are used to switch a specific music parameter. The fourth level is a specific music clip, that is, the performance content (for example, notes or pitches). Figure 3 is a schematic diagram of the configuration of the music module according to some embodiments of this specification. Figure 3 As shown, the first level is used to switch playing styles, for example, Figure 2 The performance styles of Rock 1, Rock 2, Disco 1, Disco 2, etc. correspond to Figure 3 The middle performance style is Disco_A, Disco_B, Rock_A, etc. The second level is used to switch the performance density, for example, Figure 2 The sparse, medium and dense in the text correspond to Figure 3 The third level is used to switch the playing intensity. Figure 2 The light, medium and heavy levels correspond to Figure 3 Intensity_1, Intensity_2, Intensity_3 in .

[0067] In some embodiments, each sub-level content in the Switch container needs to be mapped and configured with the Switch Group. For example, Switch Groups representing playing style, playing density, and playing intensity can be created respectively, and added to the Switch containers of the first level, second level, and third level respectively. The Switch Group of playing style contains Switches such as Rock 1 and Rock 2; the Switch Group of playing density contains Switches such as Sparse, Medium, and Dense; the Switch Group of playing intensity contains Switches such as Light, Medium, and Heavy.

[0068] In some embodiments, the first-level Switch container includes an auxiliary MIDI file, which does not contain a music clip and is used to stop the performance of other music clips in MIDI, also known as a silent MIDI clip. For example, if the MIDI clip selected by the user includes different pitches C1, C2, etc., when a drum set receives a note with pitch C1 in the MIDI clip, the audio sample of the bass drum corresponding to pitch C1 in the drum set will be output. If a stop performance instruction is received when outputting the audio sample of the bass drum, the sound of the bass drum will be stopped directly in the traditional intelligent drum system, instead of stopping the performance after the bass drum sound is played. In the present application, by switching to a silent MIDI clip, the audio sample of the bass drum corresponding to pitch C1 will be played, and the pitch C2 after C1 in the MIDI clip selected by the user will not continue to be played. Therefore, when the user executes the stop performance operation, the unplayed part of the bass drum audio sample (for example, the tail sound) will not disappear immediately, and there is no need to destroy the original natural attenuation in the audio sample, so that the performance of the end of the performance is more real and natural, just like a real drummer stopping the performance.

[0069] In some embodiments, the hierarchical structure further includes a fourth level, which is a child of the third level, and the fourth level may include a playlist of specific performance content. Each MIDI clip is placed in a playlist, and adding the playlist may achieve "loop playback" of the MIDI clip. For example, the descriptive information of the MIDI file music performance may include "Rock 1 - Sparse - Light ( Figure 3 The playlist of this MIDI file can be configured to be placed together with "Rock 1 - Sparse - Medium" and "Rock 1 - Sparse - Heavy". Figure 3 In the path "Rock_A——Density_1". It should be noted that Figure 3 In this example, we only take the creation of different rhythm types of Rock and Disco as an example, and only expand the subset of Rock 1 (Rock_A) for display. When more playing styles, playing densities and playing strengths are needed, you can follow this example.

[0070] In some embodiments, the music parameters corresponding to the first level, the second level and the third level may be playing style, playing intensity and playing density, respectively. In other words, the second level may be used to switch playing intensity, and the third level may be used to switch playing density.

[0071] In some embodiments, the music parameters corresponding to the first level, the second level, and the third level may be playing intensity, playing density, and playing style, respectively, that is, the first level may be used to switch playing intensity, the second level may be used to switch playing density, and the third level may be used to switch playing style. The music parameters corresponding to different levels may be adjusted according to actual needs, and will not be repeated here.

[0072] The configuration of the music module 134 also includes the logic of configuring the conversion phase for the first level, the second level, and the third level in the music module 134, that is, when switching from one Switch to another Switch, the timing of ending the old content and entering the new content, and the configuration of the entry position. For example, in the first level (for example, playing style switching), the configuration of the timing of switching from one Switch to another Switch can be selected as Entry Cue, and Entry Cue refers to the exact moment or signal point when the audio content starts playing during the performance. In other words, the switching of the playing style from one style to another is based on a specific moment, for example, the moment when the music module 134 receives the user playing style switching instruction sent by the game engine is the exact moment or signal point when the playing style switches from one style to another. In the second and third levels, the configuration of the timing of switching from one Switch to another Switch (for example, switching of playing intensity or playing density) can be selected as "Same Time as Playing Segment", which means that the current audio segment (segment) ends or changes synchronously at a specific point thereof. In this way, the triggering of audio events is combined with the natural end or rhythm point of the currently playing content. The purpose of this is to allow Smart Drummer to play from the beginning only when starting to play, and any switch during the period can take effect immediately and continue the previous beat. For example, if the end of the second measure in MIDI file 1 has been played before the switch, then after switching MIDI file 2, the third measure in MIDI file 2 will continue to play instead of returning to the beginning of the first measure. This is crucial for a smooth ensemble and performance experience, especially when you want to change a rhythm pattern during a certain section of a song, the original beat of the music will not be disrupted due to the switch.

[0073] In some embodiments, the configuration of the music module 134 further includes establishing an association between the music module and the sound effect module, so that the audio sample of the sound effect module 136 is driven by the music clip sent by the music module 134. For example, in the first level of the music module 134, the target of receiving the MIDI file is set as the first level of the sound effect module 136, so that the association between the music module 134 and the sound effect module 136 is established, so that the specific content trigger of the sound effect module 136 is driven by the MIDI music clip.

[0074] The sound effect module 136 is used to provide a plurality of preset audio data sets and provide a switching function between different audio data sets. In some embodiments, the sound effect module 136 is configured to receive the target music segment output by the music module 134, and in response to receiving the target music segment, play the target audio data set based on the target music segment.

[0075] Each preset audio data set includes audio samples of different drum parts with the same sound effect style. Different drum parts with the same sound effect style can also be called a drum set.

[0076] A drum set includes a combination of multiple different types of drums and / or cymbals, such as a snare drum, a bass drum, a tom-tom, a hi-hat, a crash cymbal, a stack cymbal, a Chinese cymbal, a splash cymbal, a sub-bass drum, etc. Each drum set corresponds to a sound effect style, that is, the sound timbre produced by different drum parts in the drum set will make the music played by these drum parts have the corresponding sound effect style.

[0077] Different types of drums or cymbals can produce different timbre sounds. For example, a snare drum produces a crisper sound with a more obvious spring sound, while a kick drum has a deeper sound.

[0078] The same type of drum or cymbal can produce sounds with different timbres. The same type of drum or cymbal can produce sounds with different timbres through different playing techniques, striking positions, and equipment. For example, striking in the center of the drumhead usually produces a deeper, more physical sound; striking near the edge of the drum usually produces a brighter, higher-frequency sound. For another example, using different drumsticks or brushes to strike the same drum will produce sounds with different timbres. For another example, using a brush instead of a drumstick will produce a soft and gentle sound, which is often used in jazz. For another example, different hammering methods and force control will significantly change the timbre.

[0079] The sound of one timbre emitted by a drum part (e.g., a drum or a cymbal) can be taken as one audio sample. The same type of drum parts (e.g., a drum or a cymbal) can emit sounds of different timbres, and thus multiple independent audio samples can be generated. For example, the audio sample corresponding to a snare drum may include an audio sample generated by hitting the edge and an audio sample generated by hitting the center, and these two audio samples correspond to two timbres.

[0080] In some embodiments, the sounds produced by different drums or cymbals can be classified into different sound effect styles according to the timbre characteristics of different types of drums or cymbals, that is, the timbre of the sound of different drums or cymbals is suitable for different sound effect styles. The sound effect styles may include, for example, pop style, jazz style, heavy metal style, rock style, hip-hop style, etc. In some embodiments, the sounds produced by different types of drums or cymbals can be classified into different drum sets according to the sound effect styles. For example, the pop style may include drum set 1 (snare drum 1, bass drum 1, tom 1, hi-hat 1, crash cymbal 1, and roll cymbal 1), and the timbre of all drums or cymbals in drum set 1 is suitable for the pop style; the jazz style may include drum set 2 (snare drum 2, bass drum 2, tom 2, hi-hat 2, crash cymbal 2, and roll cymbal 2), and the timbre of all drums or cymbals in drum set 2 is suitable for the jazz style; the heavy metal style may include drum set 3 (snare drum 3, bass drum 3, tom 3, hi-hat 3, crash cymbal 3, and roll cymbal 3), and the timbre of all drums or cymbals in drum set 3 is suitable for the heavy metal style.

[0081] In some embodiments, different drum sets may be named using sound effect styles, for example, drum set 1 is a pop style; drum set 2 is named a jazz style; and drum set 3 is named a heavy metal style.

[0082] In some embodiments, the sound effect module 136 can be built using a Wwise engine.

[0083] Figure 4 Schematic diagram of the configuration of the sound effect module according to some embodiments of this specification. Figure 4 As shown in the figure, import the audio samples of each drum set (i.e., the preset audio data set) into the Actor-Mixer Hierarchy of the Wwise engine, which is the sound effect module. Use the music style to name each drum set. For example, the drum set name field contains Disco, Funk, and Rock, which respectively indicates that this is a set of drum set sounds suitable for the Disco, Funk, and Rock styles. In actual application, you can define the naming rules according to the actual situation.

[0084] Figure 4The first layer of the structure, Virtusllnstrument-Drum (the parent), is a Switch container; the Switch container is also called a Switch Group, which is used to organize and manage a group of related switches. Each Switch corresponds to a drum set. By selecting different switches, you can switch different sound effects or music. The second layer is a number of mixed containers representing different timbre drum sets (for example, Virtusllnstrument-Drum-Rock, Disco, Fun). The third layer is the audio samples of each specific drum of the different timbre drum sets in the mixed container. These three layers correspond to Figure 2 The "Switch container with multiple drum kits", "Acoustic drums 1 (mixed container)" and "Triangles in Acoustic drums 1 (audio samples)" in the . The maximum number of sounds in a mixed container can be limited to 1. In some embodiments, the number of sounds of Hihat in the Rock drum kit is 2, including the audio samples corresponding to HihatOpen and HihatClose. Adding such a mixed container to the Hihat is to achieve that whenever the closed cymbal is triggered, the adjacent triggered open cymbal in front of it is interrupted, which is more in line with the operating mechanism of the real hihat in reality.

[0085] After importing the audio samples, you need to make three configurations for the first-level Switch container, namely the allocation of specific Switches, the mapping of MIDI pitch and audio samples, and the mapping of MIDI performance intensity and audio sample volume.

[0086] First, the specific configuration of Switch includes creating a Switch Group for switching drum sets, that is, for switching between different drum sets. Each Switch in the Switch Group is assigned to a corresponding set of drum set sounds. The name of each Switch can be named according to the playing style of the drum set. In this way, when the program calls the name of each Switch in the game engine, it can switch to the corresponding drum set sound (such as Figure 2 The “Switch” in the middle right area of ​​the figure shows the drum kit sound.

[0087] The mapping of the pitch of the music clip and the audio data set includes mapping different types of drums or cymbals in each drum set to separate pitches, and establishing a mapping relationship between the pitch in the music clip and the audio sample corresponding to each drum set through the pitch. Different types of drums or cymbals in each drum set correspond to separate pitches, and the same type of drums or cymbals correspond to the same pitch. Similarly, different pitches in the music clip correspond to different timbres or types of drums, and the same timbre (i.e., drums or cymbals) is at the same pitch. The correspondence between the drums and pitches in each drum set is configured to be the same as the correspondence between the pitches and drums in the music clip, thereby establishing a mapping relationship between the pitch of the music clip (e.g., MDII pitch) and the audio data set (e.g., audio samples). For example, if D1 in the first MIDI file is configured to correspond to a snare drum, then D1 in all other MIDI files must also be configured to correspond to a snare drum or a timbre with a snare-like function (e.g., a clap timbre in an electronic drum), and cannot be other, such as a bass drum, a hi-hat, etc. Similarly, the pitch of the snare drum in each drum set is also D1. Based on the mapping relationship between the pitch of the music clip and the audio data set, the different drums or cymbals in a drum set can be driven by the music clip to play the corresponding audio samples. For example, when playing the D1 pitch or note in the MIDI file, the sound effect module will output the audio sample corresponding to the snare drum with the same pitch D1 in the drum set.

[0088] The mapping of the playing intensity of a music clip and the volume of an audio data set refers to the corresponding relationship between the playing intensity of the music clip and the volume of the audio sample. In some embodiments, the mapping of the playing intensity of a music clip and the volume of an audio data set can be represented by a function, a lookup table, etc. The mapping of the playing intensity of a music clip and the volume of an audio data set can be linear or nonlinear. For example, a real-time parameter control (RTPC) curve describing the relationship between the volume of a sample and the MIDI playing intensity can be configured in the first-layer Switch container. The specific curvature and slope of the RTPC curve can be configured as needed. The greater the playing intensity of the music clip, the greater the volume of the playing audio data set, and vice versa. This is because the intensity of most musical instruments needs to change dynamically when playing, and each note in each music clip must also contain a playing intensity information. When the drum needs to be played intensively, it is often by adding one or more additional small-intensity taps to the light beats before and after the heavy beat. Therefore, adding this configuration can better restore the originally designed performance effect in the music clip, especially for the intensively played passages.

[0089] The event trigger module 132 is used to control audio output. In some embodiments, the event trigger module 132 can be triggered to perform related audio control actions after receiving a control instruction from the game engine 120, for example, to control the start and stop of audio playback, the number of beats (i.e., BPM), the MIDI playback rate, etc. In some implementations, the event trigger module 132 includes an output parameter adjustment unit. The output parameter adjustment unit can be used to adjust audio output parameters. Audio output parameters include audio playback rate, audio playback speed, music clip playback speed (i.e., MIDI playback rate), etc. Beats per minute (BPM) refers to the number of beats played per minute (i.e., audio playback rate), which is used to describe the speed of music. The larger the value, the faster the music speed. Audio playback speed refers to the speed of an audio file or stream during playback. MIDI playback rate refers to the trigger interval of different audio samples. The smaller the trigger interval between two adjacent audio samples, the greater the MIDI playback rate. There is a direct relationship between MIDI playback rate and BPM and audio playback speed. When the MIDI playback rate increases, the BPM and audio playback speed will also increase.

[0090] In some embodiments, the output parameter adjustment unit can adjust the BPM by adjusting the MIDI playback rate based on the corresponding relationship between the MIDI playback rate and the BPM. By modifying the MIDI playback rate in real time, the effect of real-time adjustment of the BPM is achieved. This method of modifying the MIDI playback rate is different from directly modifying the audio playback rate. Modifying the MIDI playback rate actually modifies the trigger interval of audio materials such as the bass drum and the snare drum, and does not change the playback rate of the audio material itself, and does not affect the sound quality. In some embodiments, the corresponding relationship between the MIDI playback rate and the BPM can be linear, for example, the ratio between the MIDI playback rate and the BPM is a constant. This constant is related to the setting of the audio playback system.

[0091] In some embodiments, the output parameter adjustment unit can obtain the target BPM input by the user through the user terminal, and determine the target MIDI playback rate based on the target BPM and the correspondence between the MIDI playback rate and the BPM, so that the BPM can be adjusted to the target BPM by adjusting the MIDI playback rate to the target MIDI playback rate.

[0092] In some embodiments, BPM adjustment can be achieved through RTPC technology. RTPC (Real-Time Parameter Control) is a technology for dynamic audio parameter adjustment, which allows audio designers and developers to adjust audio parameters according to real-time game status or user interaction to achieve a more immersive and interactive audio experience. RTPC technology can instantly adjust audio properties according to in-game variables (such as character speed, height, health, etc.).

[0093] RTPC (Real-Time Parameter Control) is a type of control information that can affect the parameters in the Wwise engine (for example, MIDI playback rate) through the control source. It consists of the control source name and the corresponding value. For example, the control source can be a variable in the game (such as the character's speed, height, health, etc.), and the parameters in the Wwise engine include the MIDI playback rate.

[0094] In some implementations, the control source may be a BPM value (ie, target BPM) set by a user within the game.

[0095] In some embodiments, the output parameter adjustment unit can be constructed by a hierarchical structure. The hierarchical structure includes a first level and a second level. The first level is used to set the mapping relationship between BPM and MIDI playback rate. The mapping relationship between BPM and MIDI playback rate can be a linear relationship. For example, the ratio between BPM and MIDI playback rate can be set to a constant of 100, 120, etc. The second level is used to configure RTPC based on the mapping relationship between BPM and MIDI playback rate, so that the playback speed of the MIDI music clip (i.e., MIDI playback rate) can be determined according to the target BPM value based on the mapping relationship between BPM and MIDI playback rate.

[0096] like Figure 5 As shown, Figure 5 This is a schematic diagram of controlling BPM based on RTPC technology according to some embodiments of this specification. For example, first set the ratio between BPM and MIDI playback rate to 100 on the first level. Other values ​​are also acceptable. It is recommended to set it to 100 for ease of calculation. Then configure RTPC on the second level. RTPC represents the mapping relationship between BPM (horizontal axis) and MIDI playback rate (vertical axis). The value range of RTPC can be set to 30 to 240 according to actual needs. The value of RTPC will be sent by the game engine based on user input or game engine default settings when the game is running. For example, the value of RTPC can be based on the BPM value set by the user (i.e., the target BPM value, for example, Figure 2Through RTPC configuration, you can set the horizontal and vertical coordinates of the RTPC to be 0.3 when the X axis is equal to 30, 2.4 when the X axis is equal to 240, etc., that is, the ratio between BPM (horizontally) and MIDI playback rate (vertically) is 100.

[0097] If the user sets the BPM value (the horizontal axis of RTPC, i.e. the target BPM) to 40, the vertical axis of RTPC is determined to be 0.4, that is, the MIDI playback rate is 0.4. Previously, we have set the overall BPM of the first level to 100, so the actual playback speed is 0.4*100, which is equal to 40BPM, which is the BPM set by the user. In this way, the BPM control has been configured.

[0098] In some embodiments, the Source of MIDI Clip Tempo in the music module 134 must ensure that Hierarchy is selected so that the 100BPM set in the BPM control will take effect and ensure that the actual BPM of the performance is consistent with the BPM set by the user. MIDI Clip Tempo refers to the speed value associated with the MIDI clip, which defines the rhythm when playing MIDI information.

[0099] In some embodiments, the event trigger module 132 includes a music parameter conversion unit. The music parameter conversion unit is used to realize the conversion of music parameters (e.g., performance density, performance intensity, etc.). The music parameter conversion unit is used to receive music parameter information set or input by a user in the game engine, determine the target music parameter based on the music parameter information, and send the target music parameter to the music module 134 to determine the target music clip (e.g., target MIDI file), and the target music clip (e.g., target MIDI file) includes the target music parameter. In some embodiments, the music parameter conversion unit can be integrated into the music module 134. In some embodiments, the music parameter conversion unit can be another module independent of the event trigger module 132.

[0100] In some embodiments, the music parameter conversion unit may obtain music parameter information set or input by the user, and the music parameter information may include identification information of the music parameter (e.g., the name of the performance style), a numerical range corresponding to the music parameter (e.g., an RTPC value), or a numerical value corresponding to the music parameter. The music parameter conversion unit may determine the target music parameter based on the music parameter information input by the user, and send the determined target music parameter to the music module 134, and the music module 134 determines the target music segment (e.g., a target MIDI file) based on the received target music parameter. The target music segment (e.g., a target MIDI file) is the MIDI file selected by the user to be played.

[0101] In some embodiments, for the performance style, the user may input the name of the performance style, and the music parameter conversion unit may determine the target performance style based on the name of the performance style input by the user.

[0102] In some embodiments, for music parameters such as performance density or performance intensity, the user can input a numerical value corresponding to the performance density or performance intensity, and the music parameter conversion unit can determine the target performance density and the target performance intensity based on the numerical value input by the user. In some embodiments, the music parameter conversion unit can store a mapping relationship between the numerical value and the performance density or performance intensity, and the music parameter conversion unit can determine the target performance density and the target performance intensity based on the mapping relationship and the numerical value input by the user.

[0103] In some embodiments, for music parameters such as performance density or performance intensity, the user can input the level corresponding to the performance density or performance intensity (for example, "light, medium, heavy" corresponding to the performance intensity or "sparse, medium, dense" corresponding to the performance density, etc.), and the music parameter conversion unit can determine the target performance density and target performance intensity based on the level input by the user.

[0104] In some embodiments, the selection or switching of music parameters can be controlled by RTPC technology. The RTPC value comes from the numerical value corresponding to the music parameter input by the user in the game engine. The event trigger module can determine the corresponding target music parameter based on the RTPC value, and the determined target music parameter constitutes the target music clip selected by the user (for example, the target MIDI file).

[0105] For example, you can create an RTPC associated with each of the two Switch Groups representing the playing density and playing strength. The selection or switching of the Switch in these two Switch Groups does not send the Switch directly, but sends the value of the RTPC, and then converts the RTPC value into a Switch (corresponding to the specific playing density or playing strength) in the Wwise engine. Not sending the Switch directly can make program development much simpler, that is, the X / Y controller can be used through simpler development, and this is also conducive to possible expansion and flexibility in the future.

[0106] Further example, Figure 6 1 is a schematic diagram of selecting or switching the performance density based on RTPC according to some embodiments of the present application. As shown in the figure, if the RTPC value range corresponding to the performance density is set to 0 to 100, the RTPC value can come from the user in the game engine through the X / Y controller (such as Figure 8 The value corresponding to the performance density input on the X-axis of the Figure 6 After configuration, when the X-axis transmission value of the X / Y controller is in the range of 0 to 33.333, the Switch representing the playing density will switch to "Sparse (Density_1)", when it is above 33.333 to 66.666, it will switch to "Medium (Density_2)", and when it is above 66.666 to 100, it will switch to "Density (Density_3)". Similarly, the Switch representing the playing intensity is also associated with the RTPC representing the Y-axis value. For example, the RTPC value range of the corresponding playing density can be set to 0 to 100, and the RTPC value can come from the user in the game engine through the X / Y controller (such as Figure 8 The value corresponding to the performance density input on the Y axis of the Figure 6 After configuration, when the Y-axis transmission value of the X / Y controller is in the range of 0 to 33.333, the Switch representing the playing intensity will switch to "Light (Intensity_1)", when it is above 33.333 to 66.666, it will switch to "Medium (Intensity_2)", and when it is above 66.666 to 100, it will switch to "Heavy (Intensity_3)".

[0107] In some embodiments, the event trigger module 132 further includes a control unit. The control unit is used to receive operation instructions input by the user sent by the game engine, and perform audio-related actions based on the operation instructions, such as playing a certain audio, stopping a certain audio, resetting certain parameters, etc.

[0108] In some embodiments, the control unit may include events for starting and stopping playback. The target for starting playback includes the Switch container at the highest level of the music module. The target for stopping playback is the Switch corresponding to the MIDI file without audio clips in the music module. In this way, when stopping, it can be ensured that the tail of the instrument is played naturally instead of being forcibly interrupted.

[0109] The game engine 120 is responsible for implementing the game functions of the terminal, such as rendering 3D or 2D graphics, playing sound effects and music, processing user input, and executing game logic. The game engine includes a user interface, through which the user implements the game functions and / or audio playback functions. For example, the game engine allows the user to operate, navigate, perform actions, etc. in the game environment.

[0110] The user interface includes an interactive control module, which may include a playback control module 122 , a music selection module 124 , and a sound effect selection module 126 .

[0111] The playback control module 122 is used to receive the user's operation instructions and send the user's operation instructions to the event trigger module, music module and / or audio module of the audio engine to perform corresponding operations. For example, the playback control module 122 may include a play button, a stop button, a BPM selector, etc. For example, Figure 7 is a schematic diagram of a user interface according to some embodiments of this specification. Figure 7 As shown, the user interface may include a play button 708 , a stop button 710 , and a BPM selector 712 .

[0112] When the event triggering module 132 in the audio engine 130 receives the operation instruction of the user clicking the play button 708 sent by the game engine 120, the event of starting to play is triggered. In some embodiments, after clicking the play button to enter the play state, the play button will become a "stop button".

[0113] When the event triggering module 132 in the audio engine 130 receives the operation instruction of the user clicking the stop button 710 sent by the game engine 120, the stop playback event is triggered. In some embodiments, after clicking the stop button to enter the stop state, the stop button will become a "play button".

[0114] The BPM selector 712 is used by the user to determine the desired target BPM. The game engine can send the target BPM determined by the user to the event trigger module (e.g., the output parameter adjustment unit), and the event trigger module adjusts the MIDI playback rate based on the target BPM determined by the user to adjust the BPM to the target BPM. For more detailed descriptions of the event trigger module adjusting the MIDI playback rate based on the target BPM input by the user, see the description in other parts of the specification.

[0115] For example, Fig. 9 Schematic diagram of a BPM selector according to some embodiments of the present application. The BPM value range is consistent with the range set in the audio engine, for example, 30 to 240. When the user moves the cursor or clicks the plus or minus button, the value is changed and sent to the audio engine at the same time. The audio engine will use the user input to control the playback rate multiple (such as Figure 6 The rectangular box on the left of the BPM selector can represent the ratio between BPM and MIDI playback rate, which can be set to a default value, for example, any value between 80 and 120, such as 100.

[0116] The music selection module 124 is used to receive music parameter information input by the user, such as performance style information, performance density information, performance intensity information, etc., and send the music parameter information input by the user to the music module 134 or the event trigger module 132 (e.g., the music parameter conversion unit) in the audio engine 130 to determine the target music segment (e.g., the target MIDI file). In some embodiments, the music selection module 124 may include a voice input button, and the user can input the music parameter information by voice input.

[0117] In some embodiments, the music selection module 124 includes a style selector and an X / Y controller. For example, Figure 7 A style selector 702 and an X / Y controller 704 are shown.

[0118] The style selector 702 is an input device for selecting a performance style. In some embodiments, the style selector can select various main performance styles such as rock and disco in the form of tabs, and the main performance style will include sub-performance styles. For example, rock will include rock 1 and rock 2. The name of the performance style can be named as needed according to actual conditions, for example, punk rock, metal rock, funk rock, etc. When a specific sub-style (i.e., the target performance style) is selected, the corresponding Switch name of the sub-style is sent to the audio engine, and the music module in the audio engine, for example, "Switch container containing multiple MIDI clips" will switch to the sub-performance style. In some embodiments, the style selector can select the target performance style in the form of voice.

[0119] In some embodiments, a default sub-style may be set to ensure that a style is selected as the default style even if the user does not make any selection.

[0120] The X / Y controller 704 is an input device for controlling two different parameters (e.g., playing density and playing intensity). For example, the X-axis of the X / Y controller can be used to control playing density, and the Y-axis can be used to control playing intensity. As the X-axis value increases, the playing density increases; as the Y-axis value increases, the playing intensity also increases.

[0121] In some embodiments, the X / Y controller can be implemented by a touch screen. The user can directly drag a finger on the screen of a mobile phone, tablet, touch display, etc. to control it. Figure 8 As shown, Figure 8 It is a schematic diagram of an X / Y controller shown in some embodiments of the present application. The X / Y controller may include a control point P. The control point P can be moved on the interface of the X / Y controller by the user's control, and correspondingly there will be corresponding coordinates at each position, namely, X-axis values ​​and Y-axis values. The X-axis values ​​and Y-axis values ​​can respectively represent the values ​​corresponding to the performance density and the performance intensity. The value ranges of the X-axis and the Y-axis must be the same as the value range set by the event trigger module in the audio engine, for example, 0 to 100. Whenever the position of the control point P is moved, the X-axis and Y-axis values ​​are sent to the audio engine in real time in the form of RTPC, respectively, to control the Switch for switching the performance density and the Switch for switching the performance intensity.

[0122] In some embodiments, the default position of the control point P can be set at the center of the X / Y controller interface. In some embodiments, each time the user adjusts the position of the control point P, the control point P can return to the default position in real time, facilitating the user to adjust the position of the control point P next time.

[0123] In some embodiments, the user can be prompted through text, voice, etc. to "move the control point horizontally to adjust the performance density (i.e., performance complexity), the further to the right the more complex the performance; move the control point vertically to adjust the performance strength, the further up the performance the stronger the performance."

[0124] The sound effect selector is an input device for selecting the style of sound effects. For example, Figure 7 The sound effect selector 706 shown. In some embodiments, the sound effect selector can select various sound effect styles such as rock, disco, etc. in the form of tabs. When a specific sound effect style (i.e., the target sound effect style) is selected, the corresponding Switch of the sound effect style is sent to the sound effect module in the audio engine. For example, a "Switch container containing multiple drum sets" will switch to the drum set corresponding to the sound effect style. In some embodiments, the sound effect selector can select the target sound effect style in the form of voice. It is basically the same as the previous style selector, the only difference is that it usually does not require 2 levels, but 1 level is enough. Please refer to the style selector for details and will not be repeated here.

[0125] The memory may be used to store system data and / or instructions, such as preset music clips, preset sound effect data sets, a mapping relationship between audio playback rate and MIDI playback rate, a mapping between music clip pitch and audio data set, a mapping between music clip pitch and audio data set, etc. The memory may include a storage module. In some embodiments, the storage module may be integrated into a music module, a sound effect module, etc.

[0126] Figure 2 1 is a schematic diagram of a specific embodiment of an audio playback system according to some embodiments of this specification. Figure 2 As shown, the game engine includes a play button, a stop button, a style selector, an X / Y controller, a timbre selector, and a BPM slide selector. The style selector can be displayed in the user interface in the form of a tab. The user can select the desired performance style based on the name of the performance style in the tab. For example, the rock style also includes subsets, rock 1, rock 2..., and the disco style includes subsets, disco 1, disco 2..., and so on. When the user clicks the rock style, the subset under the rock style will be expanded in the user interface for the user to select. The X / Y controller includes an X-axis value and a Y-axis value, and the user can input the performance density and performance intensity information, i.e., the RTPC value, through the X-axis value and the Y-axis value. The timbre selector (also called the sound effect selector) can be displayed in the user interface in the form of a tab, for example, acoustic drum 1, acoustic drum 2, electric drum 1,... The user can select the drum set of the sound effect style by clicking the corresponding position in the user interface.

[0127] When the user enters the playing style information through the style selector, the game engine can call the name of each playing style Switch (i.e. Figure 2When the user inputs the performance force information and the performance density information through the X / Y controller, the audio engine can convert the RTPC value including the performance density information input by the user into the name of the Switch corresponding to the performance density (i.e. Figure 2 The music module can switch to the MIDI clip of the corresponding performance density, and the audio engine can convert the RTPC value input by the user including the performance intensity information into the name of the Switch corresponding to the performance intensity (i.e. Figure 2 In the above method, the target MIDI music clip can be switched from the Switch container containing multiple MIDI music clips. The target MIDI music clip is the music clip selected by the user through the style selector and the X / Y controller.

[0128] When the user enters the sound style information through the sound color selector, the game engine can call the name of each sound style Switch (i.e. Figure 2 The sound effect module can switch to the corresponding sound effect style. The sound effect module contains multiple sets of drum set switches, each of which includes multiple audio samples. The sound effect module switches to the corresponding sound effect style to determine the audio sample set of the drum set selected by the user.

[0129] When the user inputs the desired BPM value (i.e., RTPC) through the BPM slide selector in the game engine, the music module can determine the MIDI playback rate and add the determined MIDI playback rate to the determined target MIDI segment and send it to the sound effect module. The sound effect module can switch to the Switch for the corresponding sound effect style according to the sound effect style selected by the user. When the sound effect module receives the target MIDI segment sent by the music module, the sound effect module can output the audio sample set of the drum set selected by the user based on the target MIDI segment.

[0130] Fig.10 Schematic diagram of an audio playback method according to some embodiments of this specification. The audio playback method can be executed by the processing device 112 or the electronic device 1200. The audio generation method 1000 includes the following steps.

[0131] Step 1002 , obtaining a first input inputted through a user interface. In some embodiments, step 1002 may be executed by the music selection module 124 of the game engine 120 in the processing device 112 or by the obtaining module 1102 in the audio playback apparatus 1100 .

[0132] The first input includes music parameter information set by the user. The music parameter information may include performance style information, performance density information, performance intensity information, etc. For example, the music parameter information may include an identifier of the performance style (e.g., name), an identifier corresponding to the performance density (e.g., numerical value or level), an identifier corresponding to the performance intensity (e.g., numerical value or level, etc. In some embodiments, the user may select a desired performance style from a plurality of preset performance styles through a style selector of the user interface to generate performance style information. In some embodiments, the user may input numerical values ​​corresponding to the performance density and performance intensity through an X / Y controller of the user interface to generate performance density information and performance intensity information. In some embodiments, the user may input music parameter information through voice of the user interface. For more descriptions of music parameter information, user interface, etc., please refer to the descriptions in other parts of the specification, for example, Figure 1 , 2 as well as Figure 7-9 Detailed description.

[0133] Step 1004 , obtaining a second input inputted through the user interface. In some embodiments, step 1004 may be executed by the sound effect selection module 126 of the game engine in the processing device 112 or by the obtaining module 1102 in the audio playback apparatus 1100 .

[0134] The second input includes sound effect data information set by the user. The sound effect data information may include music style information. For example, the sound effect data information may include an identifier (e.g., name) of the sound effect style. In some embodiments, the user may select a desired sound effect style from a plurality of preset sound effect styles through a sound effect selector of the user interface to generate the sound effect data information. In some embodiments, the user may input the sound effect data information (e.g., the name of the sound effect style) by voice through the user interface. For more descriptions of the sound effect style and the user interface, please refer to the descriptions in other parts of the specification, for example, Figure 1 , 2 as well as Figure 7-9 Detailed description.

[0135] Step 1006 , determining a target music clip from a plurality of preset music clips based on the first input. In some embodiments, step 1006 may be executed by the play control module 112 of the game engine in the processing device 112 or by the music clip determination module 1104 in the audio playback apparatus 1100 .

[0136] In some embodiments, each music clip in the plurality of preset music clips may have a plurality of different music parameters, such as descriptions of playing style, playing density, playing intensity, etc. Different combinations of music parameters may produce different music performances. In some embodiments, the plurality of music parameters may be saved in the format of a MIDI file, i.e., each music clip is stored in a MIDI file, also referred to as a MIDI music clip or a MIDI clip.

[0137] In some embodiments, a target performance style corresponding to the target music clip can be determined based on the performance style information in the first input. A target performance density corresponding to the target music clip can be determined based on the performance density information in the first input. A target performance intensity corresponding to the target music clip can be determined based on the performance intensity information in the first input. Further, based on the target performance style, target performance density, and target performance intensity, a target music clip matching the target performance style, target performance density, and target performance intensity can be screened out from a plurality of preset music clips.

[0138] In some embodiments, for example, a first candidate music segment with a target performance style can be screened out based on the performance style information (or target performance style) in the first input, and then a second candidate music segment can be screened out from the first candidate music segment based on the performance density information (or target performance density) in the first input, and finally, the target music segment can be screened out from the second candidate music segment based on the performance intensity information (or target performance intensity) in the first input.

[0139] In some embodiments, for example, a first candidate music clip having a target playing style may be first selected based on the playing style information (or target playing style) in the first input, a second candidate music clip may be selected from the first candidate music clip based on the playing intensity information (or target playing intensity) in the first input, and finally a target music clip may be selected from the second candidate music clip based on the playing density information (or target playing density) in the first input. The specific method of determining the target music clip may be related to the hierarchical construction of the music module 134 in the audio engine 130. For example, the music module 134 includes a first level, a second level and a third level. If the first level includes a switch group corresponding to the performance density, the second level includes a switch group corresponding to the performance intensity, and the third level includes a switch group corresponding to the performance style, then the music module 134 can first filter out a first candidate music segment with a target performance density based on the performance density information (or target performance density) in the first input, and then filter out a second candidate music segment with a target performance intensity from the first candidate music segment based on the performance intensity information (or target performance intensity) in the first input, and finally filter out a target music segment with a target performance style from the second candidate music segment based on the performance style information (or target performance style) in the first input.

[0140] In some embodiments, the performance density or the performance intensity can be divided into multiple levels. For example, the performance density can be divided into "sparse", "medium", "dense", etc., and the performance intensity can be divided into "light", "medium", "heavy", etc. The music parameter information in the first input may include the value of the performance density and the value of the performance intensity. The event triggering module 132 can determine the level of the performance density expected by the user (i.e., the target performance density) according to the mapping relationship between the value of the performance density and the level of the performance density. The event triggering module 132 can determine the level of the performance intensity expected by the user (i.e., the target performance intensity) according to the mapping relationship between the value of the performance intensity and the level of the performance intensity. For more descriptions on determining the target performance density and the target performance intensity based on the values ​​of the performance density and the performance intensity, please refer to the descriptions in other parts of the specification.

[0141] In some embodiments, the music parameter information in the first input may include a level of performance density and a level of performance intensity, and the event trigger module 132 may designate the level of performance density and the level of performance intensity in the first input as target performance density and target performance intensity, respectively.

[0142] In some embodiments, the event triggering module 132 or the music segment determination module 1104 may send the determined target music segment to the music module 134 or the output module 1108 respectively.

[0143] Step 1008, determining a target sound effect data set from a plurality of preset sound effect data sets based on the second input. In some embodiments, step 1008 may be performed by the sound effect module 136 of the game engine in the processing device 112 or by the sound effect data determination module 1106 in the audio playback device.

[0144] In some embodiments, a target sound effect data set matching the sound effect style may be determined from a plurality of preset sound effect data sets based on the sound effect style information in the second input.

[0145] Each of the plurality of preset sound effect data sets may include audio samples of different drum parts with the same sound effect style. Different drum parts with the same sound effect style may also form a drum set. Each drum part in the same drum set corresponds to an audio sample. For more descriptions of the audio samples, please refer to the detailed descriptions in other parts of this specification, such as Figure 1 and 2 and its detailed description.

[0146] In some embodiments, each preset sound effect data set may have an identifier for identifying the sound effect style corresponding to each preset sound effect data set. The identifier of each preset sound effect data set may be the name of the sound effect style corresponding to each preset sound effect data set, such as rock, disco, etc., or may be named after the category of drums having the sound effect style (e.g., acoustic drums, electronic drums, etc.). The target sound effect data set may be determined by the name of the sound effect style in the sound effect style information in the second input and the sound effect style names corresponding to the plurality of preset sound effect data sets.

[0147] Step 1010: Drive the target sound effect data set to play based on the target music clip. In some embodiments, step 1010 may be performed by the sound effect module 136 in the processing device 112 or by the output module 1108 in the audio playback device.

[0148] In some embodiments, the target music clip includes a time sequence of different notes. Each note corresponds to a pitch. The target sound effect data set includes a plurality of different audio samples, each of which corresponds to a drum part and a pitch. When the sound effect module receives a specific note in the target music clip, an audio sample of a drum part having the same pitch as the specific note will be output and played. For a detailed description of the target music clip driving the sound effect data to be played, reference can be made to the description elsewhere in the specification.

[0149] In some embodiments, a third input of the user may be obtained, and the third input may include a target playback rate (i.e., a target BPM value) set by the user. Based on the mapping relationship between the audio playback rate and the music clip playback speed (e.g., the MIDI playback rate) and the target number of beats, the target music clip playback speed (i.e., the target MIDI playback rate) is determined; and the target sound effect data set is played based on the target music clip playback speed. For the determination of the music clip playback speed, reference may be made to the detailed description in other parts of the specification.

[0150] Fig.11 Schematic diagram of an audio playback device according to some embodiments of the present specification. The audio playback device 1100 is used to execute the audio playback method 1000. The audio playback device 1100 may include an acquisition module 1102, a music segment determination module 1104, a sound effect data determination module 1106, and an output module 1108.

[0151] The acquisition module 1102 is used to acquire a first input inputted by a user through a user interface and to acquire a second input inputted by a user through a user interface. The music segment determination module 1104 is used to determine a target music segment from a plurality of preset music segments based on the first input. The sound effect data determination module 1106 is used to determine a target sound effect data set from a plurality of preset sound effect data sets based on the second input. The output module 1108 is used to drive the target sound effect data set to play based on the target music segment.

[0152] The present application also provides an electronic device for audio playback. Fig.12 As shown, the electronic device 1200 includes: a processor 1202 and a memory 1204, which are used to store a program for a method of playing music. After the device is powered on and runs the program for the method of playing music through the processor, the following steps are performed: obtaining a first input from a user through a user interface, the first input including music parameter information; obtaining a second input from a user through the user interface, the second input including sound effect data information; determining a target music clip from a plurality of preset music clips based on the first input; determining a target sound effect data set from a plurality of preset sound effect data sets based on the second input; and outputting a target sound effect data set based on the target music clip.

[0153] An embodiment of the present application provides a computer-readable storage medium storing a program for a method for playing music, wherein the program is executed by a processor to perform the following steps: obtaining a first input from a user through a user interface, wherein the first input includes music parameter information; obtaining a second input from a user through the user interface, wherein the second input includes sound effect data information; determining a target music clip from a plurality of preset music clips based on the first input; determining a target sound effect data set from a plurality of preset sound effect data sets based on the second input; and outputting a target sound effect data set based on the target music clip.

[0154] Some embodiments of the present specification also provide a computer program product, including computer instructions. When at least part of the computer instructions are executed by a processor, the present specification can be implemented. Fig.10 In some embodiments, the computer program product may only involve computer instructions, which may be carried by a storage medium or a processing device. In other embodiments, the computer program product may also be a storage medium or a processing device containing the aforementioned computer instructions. The processing device may include one or more processors and a storage medium.

[0155] In some embodiments, the processor may be a combination of one or more of the following processors: a central processing unit (CPU), an application specific integrated circuit (ASIC), an application specific instruction set processor (ASIP), a graphics processing unit (GPU), a physical processing unit (PPU), a digital signal processor (DSP), a field programmable gate array (FPGA), a programmable logic device (PLD), a programmable logic controller (PLC), a reduced instruction set computer (RISC), and a microprocessor.

[0156] In some embodiments, the storage medium may include a combination of one or more of the following: a mass storage, a removable storage, a volatile read-write memory, and a read-only memory (ROM). Exemplary mass storage may include a magnetic disk, an optical disk, a solid-state hard disk, and the like. Exemplary removable storage may include a flash disk, a floppy disk, an optical disk, a memory card, a compressed hard disk, a magnetic tape, and the like. Exemplary volatile read-write memory may include a random access memory (RAM). Exemplary random access memory may include a dynamic random access memory (DRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), a static random access memory (SRAM), a thyristor random access memory (T-RAM), and a zero capacitance memory (Z-RAM), and the like. Exemplary read-only memory may include a masked read-only memory (MROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a compact disk read-only memory (CD-ROM), and a digital multifunction hard disk read-only memory, and the like.

[0157] Although the present application is disclosed as above with a preferred embodiment, it is not intended to limit the present application. Any person skilled in the art may make possible changes and modifications without departing from the spirit and scope of the present application. Therefore, the scope of protection of the present application shall be based on the scope defined by the claims of the present application. For example, the electronic device 1200 also includes one or more input / output interfaces, a network interface, and a memory. The memory may include non-permanent memory in a computer-readable medium, a random access memory (RAM) and / or a non-volatile memory in the form of a read-only memory (ROM) or a flash memory (flash RAM). The memory is an example of a computer-readable medium. Computer-readable media include permanent and non-permanent, removable and non-removable media, and information storage can be implemented by any method or technology. The information can be a computer-readable instruction, a data structure, a module of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), random access memory (RAM) of other attributes, read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage media or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include non-transitory media such as modulated data signals and carrier waves.

[0158] For more information about each module, please refer to the relevant descriptions in other parts of the specification, which will not be repeated here. It should be understood that the system and its modules shown in this specification can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented by hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer executable instructions and / or control codes contained in a processor, such as a disk, CD or DVD-ROM, etc., and such codes are provided in the memory of a programmable device. The system and its modules of this specification can not only be implemented by hardware circuits such as ultra-large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., but can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above hardware circuits and software (for example, firmware).

[0159] It should be noted that the above description of the system and its modules is only for convenience of description and does not limit this specification to the scope of the embodiments. It is understandable that for those skilled in the art, after understanding the principle of the system, it is possible to arbitrarily combine the modules without deviating from this principle to form a subsystem connected to other modules. Or some modules may be split to obtain more modules or multiple units under the module. Such variations are all within the scope of the disclosure of this specification.

[0160] The beneficial effects that may be brought about by the embodiments of this specification include but are not limited to: (1) splitting each drum set into independent audio materials (i.e., audio samples) according to the specific timbre components such as the bass drum, snare drum, hi-hat, tom, and crash cymbal contained therein, forming multiple preset audio material sets (i.e., sound effect data sets), combining MIDI files with the sound effect data sets, and then using MIDI files to drive and organize playback. In this way, the separated drums are equivalent to musical instruments, which determine the timbre. The MIDI file is like a musical score, which determines when to trigger a specific timbre. The user is like the "conductor" or "DJ" here, who decides which drum set and which musical score to choose. This allows the smart drummer's playing style, playing density, playing intensity, BPM and other musical parameters and instrument timbre to be easily adjusted in real time, take effect in real time, and switch seamlessly. In addition, the musical parameters and playing content can be arbitrarily combined with drum sets of different timbres to enrich the audio and facilitate user operation. (2) The BPM can be modified by modifying the MIDI playback rate. The BPM can be adjusted to any value without affecting the sound quality. (3) Many MIDI clips of different styles, different rhythms, different note densities, and different playing intensities can be pre-made to enrich the performance of the smart drummer, with richer performance and choices. (4) The audio data is stored in the form of MIDI files, which is very small in size and can significantly save the consumption of additional storage resources. (5) The audio playback system can interact in the game in a simple and friendly way, with flexible control, rich performance, and low system development cost. It should be noted that different embodiments may produce different beneficial effects. In different embodiments, the beneficial effects that may be produced may be any one or a combination of the above, or any other beneficial effects that may be obtained.

[0161] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are taught in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.

Claims

1. An audio playback method, characterized in that: include: Acquire a first input inputted through a user interface, wherein the first input includes music parameter information; Acquire a second input inputted through the user interface, wherein the second input includes sound effect data information; determining a target music segment based on the first input, wherein the target music segment is described by a target music parameter; determining a target sound effect dataset based on the second input, the target sound effect dataset comprising audio samples of different drum parts having the same sound effect style; as well as The target sound effect data set is driven to be played based on the target music clip.

2. The audio playback method according to claim 1, characterized in that: The music parameter information includes performance style information, performance density information, and performance intensity information. The determining of the target music segment based on the first input includes: Determining a target playing style, a target playing density, and a target playing intensity based on the playing style information, the playing density information, and the playing intensity information; and The target music segment is determined from a plurality of preset music segments based on the target performance style, the target performance density and the target performance intensity, each of the plurality of preset music segments being described by a plurality of music parameters, and the preset music segments being stored in a digital music interface (MIDI) file format.

3. The audio playback method according to claim 1, characterized in that: The same timbre in the plurality of preset music clips is at the same pitch.

4. The audio playback method according to claim 1, characterized in that: The sound effect data information includes sound effect style information, and determining a target sound effect data set based on the second input includes: Based on the sound effect style information, the target sound effect data set is determined from a plurality of preset sound effect data sets, each of the plurality of preset sound effect data sets comprising audio samples of each drum part in a group of drum parts, and the audio samples of each drum part correspond to the same sound effect style.

5. The audio playback method according to claim 1, characterized in that: The method further comprises: Obtaining a third input inputted through the user interface, wherein the third input includes a target beat rate (BPM); Determine a target music clip playback speed based on a mapping relationship between the beat number and the music clip playback speed and the target beat number; and The target sound effect data set is played based on the target music segment playing speed.

6. The audio playback method according to claim 1, characterized in that: The user interface includes a style selector and an X / Y controller, and obtaining a first input inputted through the user interface includes: Acquiring the performance style information in the first input through the style selector; and The performance density information and the performance intensity information in the first input are obtained through the X / Y controller.

7. The audio playback method according to claim 1, characterized in that: The user interface includes a sound effect selector, and obtaining a second input inputted through the user interface includes: The sound effect data information in the second input is obtained through the sound effect selector.

8. An audio playback device, characterized in that: It comprises a processor and a memory, wherein the memory stores a computer program or a computer executable instruction, and when the computer program or the computer executable instruction is executed by the processor, the audio playback method according to any one of claims 1 to 7 is implemented.

9. A computer program product, characterized in that The invention comprises a computer program, and when at least a part of the computer program is executed by a processor, the method according to any one of claims 1 to 7 can be implemented.

10. An audio playback system, characterized in that: include: A game engine, comprising a user interface, the user interface comprising a music selection module and a sound effect selection module, the music selection module being used to obtain a first input, the first input comprising music parameter information, the sound effect selection module being used to obtain a second input, the second input comprising sound effect data information; as well as An audio engine comprises: a music module and a sound effect module, wherein the music module is configured to determine a target music clip based on the first input, wherein the target music clip is described by target music parameters; the sound effect module is configured to determine a target sound effect data set based on the second input, wherein the target sound effect data set comprises audio samples of different drum parts having the same sound effect style; and drive the target sound effect data set to play based on the target music clip.

11. The audio playback system according to claim 10, characterized in that: The music selection module includes a style selector and an X / Y controller. The style selector is used to input the performance style information in the music parameter information, and the X / Y controller is used to input the performance density information and the performance intensity information in the music parameter information.

12. The audio playback system according to claim 10, characterized in that: The user interface further includes a playback control module, the playback control module includes a beats per second (BPM) selector for obtaining a third input, the third input including a target beats per second, and the audio engine is further configured to: Determine the target music clip playback speed based on the mapping relationship between the beat number and the music clip playback speed and the target beat number; The target sound effect data set is played based on the target music segment playing speed.

13. The audio playback system according to claim 10, characterized in that: It further includes a storage module for storing a plurality of preset sound effect data sets, and the sound effect module is further configured as follows: Based on the sound effect data information, the target sound effect data set is determined from a plurality of preset sound effect data sets, each of the plurality of preset sound effect data sets comprising audio samples of each drum part in a group of drum parts, and the audio samples of each drum part correspond to the same sound effect style.

14. The audio playback system according to claim 10, characterized in that: It further includes a storage module for storing a plurality of preset music clips, and the music module is further configured as follows: The target music segment is determined based on the first input from a plurality of preset music segments, each of the plurality of preset music segments being described by a plurality of music parameters, and the preset music segments being stored in a digital music interface (MIDI) file format.

Images