Audio playing method and system and computer program product

Abstract

The invention provides an audio playing method and system and a computer program product, and relates to the technical field of audio processing, and the audio playing method comprises the steps: playing a music clip, the music clip is provided with at least one sound effect switching identifier, and the sound effect switching identifier corresponds to a sound effect set comprising at least one musical sound effect; setting a sound effect set corresponding to the sound effect switching identifier as a current sound effect set when the sound effect switching identifier is passed in the playing process of the music clip; and in response to the first trigger operation, playing the musical sound effect in the current sound effect set while playing the music clip.

Description

Technical Field

[0001] This specification relates to the technical field of audio processing, and particularly to an audio playback method, system, and computer program product. Background Art

[0002] In the audio playback scenario, it is often necessary to play other musical sound effects during the playback of a music segment. For example, in a virtual game scenario, when a player controls a virtual character to use an instrument-like weapon to attack, it is necessary to play the attack sound effect corresponding to the instrument-like weapon while playing the game background music. Since the time when the player controls the virtual character to attack is random, it is easy to make the attack sound effect generated by the instrument-like weapon during the attack form a disharmonious playback effect with the game background music, affecting the user experience.

[0003] In view of this, some embodiments of this specification provide an audio playback method, system, and computer program product, which can, when it is necessary to play musical sound effects, select the musical sound effects in the corresponding sound effect set according to the playback situation of the music segment (for example, according to the currently played part of the music segment) for playback, so that the simultaneously played musical sound effects and the music segment produce a harmonious playback effect and improve the user experience. Summary of the Invention

[0004] One or more embodiments of this specification provide an audio playback method, the method including: playing a music segment, where at least one sound effect switching identifier is set in the music segment, and the sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect; when passing through the sound effect switching identifier during the playback of the music segment, setting the sound effect set corresponding to the sound effect switching identifier as the current sound effect set; in response to a first trigger operation, playing the musical sound effects in the current sound effect set while playing the music segment.

[0005] According to the method provided by one or more embodiments of this specification, playing the musical sound effects in the current sound effect set while playing the music segment includes: randomly or based on a preset order, selecting one of the musical sound effects in the current sound effect set for playback while playing the music segment.

[0006] According to the method provided by one or more embodiments of this specification, the sound effect set includes multiple musical sound effects corresponding to the same musical attribute; the musical attribute corresponding to the current sound effect set matches the musical attribute of the currently played music segment.

[0007] According to the method provided by one or more embodiments of this specification, the musical attribute is a chord formed by the pitches corresponding to different musical sound effects according to a fixed arrangement; each of the musical sound effects corresponding to the same musical attribute included in the current sound effect set forms a harmonious interval with the currently played music segment.

[0008] According to the method provided by one or more embodiments of this specification, the sound effect set includes multiple timbre groupings with different timbre styles, and each timbre grouping includes at least one musical sound effect: in response to a first trigger operation, while playing a music segment, play the musical sound effects in the current sound effect set, including: determining a target timbre style corresponding to the first trigger operation based on the first trigger operation; while playing the music segment, randomly or based on a preset order, select a musical sound effect from a timbre grouping with the target timbre style in the current sound effect set for playing.

[0009] According to the method provided by one or more embodiments of this specification, the timbre style is classified based on the weapon category; each timbre grouping includes musical sound effects corresponding to the timbre style of the same type of weapon; determining a target timbre style corresponding to the first trigger operation based on the first trigger operation includes: determining a target weapon category corresponding to the first trigger operation based on the first trigger operation; determining a target timbre style corresponding to the target weapon category based on the target weapon category.

[0010] According to the method provided by one or more embodiments of this specification, the weapon category includes musical weapons.

[0011] According to the method provided by one or more embodiments of this specification, in response to a first trigger operation, while playing a music segment, play the musical sound effects in the current sound effect set, including: in response to the first trigger operation, display an animation segment corresponding to the first trigger operation on the user interface, and at least one sound effect trigger identifier is set in the animation segment; when passing through the sound effect trigger identifier during the playback of the animation segment, while playing the music segment, play the musical sound effects in the current sound effect set.

[0012] According to the method provided by one or more embodiments of this specification, the music segment contains an integer number of musical measures, and the sound effect switching identifier is located at the starting position of the musical measure.

[0013] According to the method provided by one or more embodiments of this specification, the music segment corresponds to the background music of the game screen, and the game screen is displayed through the user interface; the first trigger operation includes triggering a weapon and equipment in the game screen; among them, the weapon and equipment have a timbre style corresponding to its weapon category.

[0014] One or more embodiments of this specification also provide an audio playback system, which includes: a music module for playing music segments, where at least one sound effect switching identifier is set in the music segments, and the sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect; a sound effect switching module for setting the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when passing through the sound effect switching identifier during the playback of the music segment; and the music module, in response to a first trigger operation, plays the musical sound effects in the current sound effect set while playing the music segment.

[0015] One or more embodiments of this specification also provide an audio playback system, which includes: a game engine including a user interface and an input module; the user interface is used to display game screens; the input module is used to obtain a first trigger operation input; an audio engine including a music module and a sound effect switching module; the music module is used to play music segments corresponding to the game screens, where at least one sound effect switching identifier is set in the music segments, and the sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect; the sound effect switching module is used to set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when passing through the sound effect switching identifier during the playback of the music segment; and the music module, in response to the first trigger operation, plays the musical sound effects in the current sound effect set while playing the music segment.

[0016] According to the system provided by one or more embodiments of this specification, the user interface is further used to, in response to the first trigger operation, display an animation segment corresponding to the first trigger operation while displaying the game screen, where at least one sound effect trigger identifier is set in the animation segment; when passing through the sound effect trigger identifier during the playback of the animation segment, the music module plays the musical sound effects in the current sound effect set while playing the music segment.

[0017] Some embodiments of this specification also provide a computer program product, including computer instructions or a computer program, when at least a part of the computer instructions or the computer program is executed by a processor, it can implement the audio playback method provided by the embodiments of this specification. Description of the Drawings

[0018] This specification will be further described by way of exemplary embodiments, and these exemplary embodiments will be described in detail through the drawings. The same numbers in the drawings represent the same structures or steps.

[0019] Figure 1 It is a schematic diagram of an application scenario of an audio playback method shown according to some embodiments of this specification.

[0020] Figure 2 It is an exemplary flowchart of an audio playback method shown according to some embodiments of this specification.

[0021] Figure 3 It is a schematic diagram of the setting of a sound effect switching identifier shown in some embodiments of this specification.

[0022] Figure 4 It is a schematic diagram of the classification structure of a sound effect set shown in some embodiments of this specification.

[0023] Figure 5 It is a schematic diagram of the setting of another sound effect switching identifier shown in some embodiments of this specification.

[0024] Figure 6 It is a schematic diagram of the classification structure of another sound effect set shown in some embodiments of this specification.

[0025] Figure 7 It is a schematic diagram of the classification structure of another sound effect set shown in some embodiments of this specification.

[0026] Figure 8 It is a schematic diagram of the division of a musical measure shown in some embodiments of this specification.

[0027] Figure 9 It is an exemplary block diagram of an audio playback system shown in some embodiments of this specification.

[0028] Figure 10 It is an exemplary block diagram of another audio playback system shown in some embodiments of this specification. Detailed implementation manners

[0029] To more clearly illustrate the technical solutions of the embodiments of this specification, the embodiments will be introduced in detail below with reference to the accompanying drawings. Obviously, the content described below is some examples or embodiments of this specification. For those of ordinary skill in the art, without creative efforts, the technical solutions or means disclosed in this specification can also be applied to other scenarios based on these technical contents.

[0030] It should be understood that the "system", "device", "unit" and / or "module" used in this specification is a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the above words can be replaced by other expressions.

[0031] Unless otherwise specified, the technical terms describing components, elements, etc. in this specification do not specifically refer to the singular, but may also include the plural. Generally speaking, terms such as "include" and "comprise" only indicate the inclusion of the clearly identified steps, elements or components, and these steps, elements and components do not constitute an exclusive list. For example, the described method or device may also include other steps or components.

[0032] In this specification, flowcharts are used to illustrate the operation steps performed by the devices or systems of related embodiments. However, without special instructions, the order adopted when describing these steps should not be construed as a limitation on the execution order of the steps. Those of ordinary skill in the art can adjust the execution order of these steps based on the knowledge and information conveyed by the embodiments of this specification. The above adjustments include, but are not limited to, swapping the sequence, merging multiple steps, and splitting a certain step.

[0033] Audio playback is a multimedia application that plays specific audio content through playback devices such as speakers. Figure 1 It is a schematic diagram of an application scenario of an audio playback method shown according to some embodiments of this specification. As Figure 1 shown, the application scenario 100 may include a server 110, a network 120, a storage medium 130, and a terminal device 140. The server 110 and the terminal device 140 can perform data transmission through the network 120.

[0034] Among them, the server 110 can be a computer device with relatively high computing performance, and is used to generate and / or provide the audio content to be played according to user operations or other instruction information. In some embodiments, the server 110 can be a single computer device or a computing cluster composed of multiple computer devices, so as to provide more powerful computing power and more efficient response to relevant operations during the audio playback process.

[0035] The network 120 can be any form of wired or wireless network, or any combination thereof. By way of example only, the network 120 can be a wired network, a fiber optic network, a telecommunications network, an internal network, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, etc., or a combination of one or more of them. The network 120 can have multiple access points, and the server 110 and the terminal device 140 can access the network 120 through the access points.

[0036] The storage medium 130 can be used to store relevant data and / or instructions. In some embodiments, the storage medium 130 can store audio content to be played, such as music clips, musical tones, and sound effects. In some embodiments, the storage medium 130 can include one or more storage components, and each storage component can be an independent device or a part of other devices. In some embodiments, the storage medium 130 can be disposed in the server 110. In some embodiments, the storage medium 130 can be disposed in the terminal device 140. In some embodiments, the storage medium 130 can include a random access memory (RAM), a read-only memory (ROM), a mass storage device, a removable storage device, a volatile read-write memory, etc., or any combination thereof. Exemplarily, the mass storage device can include a magnetic disk, an optical disk, a solid state disk, etc. In some embodiments, the storage medium 130 can be implemented on a cloud platform.

[0037] The terminal device 140 can include, but is not limited to, electronic devices with a graphical user interface such as a desktop computer, a smart phone, a laptop computer, a game console, and a tablet computer. The terminal device 140 can obtain user operations or other instruction information through a human-machine interface device, where the human-machine interface device can be a part of the terminal device 140 or can be separated from the terminal device 140, but there is a signal connection between them. The terminal device 140 can obtain and play audio content according to the obtained instruction information. As an example, the terminal device 140 can have a signal connection with the server 110 through the network 120. The user can obtain the audio content required for playing from the server 110 and play it through the speaker of the terminal device 140 itself or a playback device connected to the terminal device 140.

[0038] It should be noted that Figure 1 The schematic diagram of the application scenario of the illustrated audio playback method is only an example. The application scenario of the audio playback method described in the embodiments of this specification is to more clearly illustrate the technical solutions of the embodiments of this specification, and does not constitute a limitation on the technical solutions provided by the embodiments of this specification. For example, in the scenario of local audio playback only through the terminal device 140, Figure 1 the illustrated server 110 and network 120 can be omitted, the storage medium 130 can be disposed in the terminal device 140, and cooperate with the application program installed on the terminal device 140 to play audio; for another example, in the scenario of audio playback through the server 110, Figure 1The terminal device 140 and the network 120 shown may be omitted. The storage medium 130 may be provided in the server 110 and cooperate with the application installed on the server 110 to play audio. As is known to those of ordinary skill in the art, with the evolution of the devices required for audio playback and the emergence of new business scenarios, the technical solutions provided in the embodiments of this specification are equally applicable to similar technical problems.

[0039] In the current audio playback scenario, it is often necessary to play other musical sound effects during the playback of a music segment. For example, in a virtual game scenario, when a player controls a virtual character to use an instrument-like weapon to attack, it is necessary to play the attack sound effect corresponding to the instrument-like weapon while playing the game background music. Since when the player controls the virtual character to attack is random, it is easy to make the attack sound effect generated by the instrument-like weapon during the attack form a disharmonious playback effect with the game background music, affecting the user experience.

[0040] In view of this, the embodiments of this specification provide an audio playback method, system, and computer program product, which can, when it is necessary to play musical sound effects, select the musical sound effects in the matching sound effect set according to the playback situation of the music segment (for example, according to the currently played part of the music segment) for playback, so that the simultaneously played musical sound effects and the music segment produce a harmonious playback effect and improve the user experience.

[0041] Figure 2 It is an exemplary flowchart of an audio playback method shown according to some embodiments of this specification. In some embodiments, Figure 2 The process 200 shown may be executed by a terminal device. For example, it may be executed by the terminal device 140 shown as Figure 1 ; or, Figure 2 The process 200 shown may also be executed by other processing devices. For example, it may be executed by the server 110 shown as Figure 1 ; or, Figure 2 The process 200 shown may be jointly executed by a terminal device and other processing devices. For example, it may be jointly executed by the terminal device 140 and the server 110 shown as Figure 1 In some embodiments, the process 200 may be implemented by an audio playback system 900 deployed on a processing device and / or a terminal device.

[0042] In some embodiments, as shown in Figure 2 The process 200 may include the following steps.

[0043] Step 210: Play a music segment, where at least one sound effect switching identifier is set in the music segment, and the sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect. In some embodiments, Step 210 can be implemented by the music module 910.

[0044] In some embodiments, the sound effect switching identifier can be a triggerable mark set by the production staff in the music segment, and this mark can be triggered during the playback of the music segment. In some embodiments, the sound effect switching identifier can be set at a specific position in the music segment according to the needs of the production staff. Figure 3 FIG. shows a schematic diagram of the setting of a sound effect switching identifier according to some embodiments of the present specification. As Figure 3 shown, for a music segment 310 with a total duration of 120 seconds, a first sound effect switching identifier 321 can be set at the position where the music segment 310 starts playing from the beginning to the Xth second (for example, taking the 40th second as an example), and a second sound effect switching identifier 322 can be set at the position where the music segment 310 starts playing from the beginning to the Yth second (for example, taking the 80th second as an example). In some embodiments, for the music segment 310 with the sound effect switching identifier set as Figure 3 shown, during the process of playing the music segment 310 starting from any position in the music segment 310, when playing the music content corresponding to the 40th second position, the first sound effect switching identifier 321 will be triggered; when playing the music content corresponding to the 80th second position, the second sound effect switching identifier 322 will be triggered. The specific setting of the sound effect switching identifier will be described later and will not be elaborated here.

[0045] In some embodiments, a sound effect switching identifier can correspond to a sound effect set, and the sound effect set can include at least one musical sound effect. In some embodiments, in order to make the simultaneously played musical sound effects and the music segment produce a harmonious playback effect, it is necessary to select musical sound effects according to the playback situation of the music segment; if only one musical sound effect can produce a harmonious playback effect with the currently played music, this musical sound effect can be directly used as a sound effect set; if there are multiple musical sound effects that can all produce a harmonious playback effect with the currently played music, these musical sound effects can all be used as a sound effect set, and a musical sound effect is selected from the sound effect set for playback when the musical sound effect needs to be played. In some embodiments, through the correspondence between the sound effect switching identifier and the sound effect set, the sound effect set can establish a correspondence with at least a part of the played music segment, thereby realizing the playback of musical sound effects that match the currently played part of the music segment and can produce a harmonious playback effect according to the playback situation of the music segment. The specific division of the sound effect set will be described later and will not be elaborated here.

[0046] Step 220: When passing through a sound effect switching identifier during the playback of a music segment, set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set. In some embodiments, step 220 can be implemented by the sound effect switching module 920. In some embodiments, step 220 can be executed synchronously during the execution of step 210, which is not limited herein.

[0047] In some embodiments, for a music segment 310 as Figure 3 shown, the first sound effect switching identifier 321 can correspond to a first sound effect set, and the second sound effect switching identifier 322 can correspond to a second sound effect set. When the music segment 310 is played to the 40th second, it passes through and triggers the first sound effect switching identifier 321, and at this time, the first sound effect set corresponding to the first sound effect switching identifier 321 is set as the current sound effect set; when the music segment 310 is played to the 80th second, it passes through and triggers the second sound effect switching identifier 322, and at this time, the second sound effect set corresponding to the second sound effect switching identifier 322 is set as the current sound effect set. As can be seen from the music segment 310 as Figure 3 shown, the first sound effect set is used as the current sound effect set when the music segment 310 is played between the 40th second and the 80th second, and the second sound effect set is used as the current sound effect set when the music segment 310 is played between the 80th second and the 120th second. In some embodiments, based on the above steps 210 and 220, each sound effect set can correspond to at least a part of the music segment, and the parts corresponding to each sound effect set in the music segment do not overlap with each other, and the switching of the current sound effect set can be realized according to the actual playback process of the music segment.

[0048] Step 230: In response to a first trigger operation, play the musical tone sound effects in the current sound effect set while playing the music segment. In some embodiments, step 230 can be implemented by the music module 910.

[0049] In some embodiments, the first triggering operation may be a triggering operation associated with musical sound effects, and the response to the first triggering operation is achieved by playing the musical sound effects. In some embodiments, the above-mentioned first triggering operation may be input by the user through a human-machine interface device such as a mouse, a keyboard, a touch screen with touch screen function, etc. When the function module for controlling the sound effect playback receives the operation instruction corresponding to the above-mentioned first triggering operation, it plays the musical sound effects. For example, in a virtual game scenario, the first triggering operation may be input by the player user by pressing the attack button on the game controller or clicking the attack virtual button in the graphical user interface. When the function module for controlling the sound effect playback receives the attack instruction corresponding to the above-mentioned first triggering operation, it plays the musical sound effects corresponding to the attack sound effect. In some embodiments, the above-mentioned first triggering operation may also be automatically generated by the application program according to the current running situation. When the function module for controlling the sound effect playback receives the operation instruction corresponding to the above-mentioned first triggering operation, it plays the musical sound effects. For example, in a virtual game scenario, the first triggering operation may be automatically generated according to the animation effect of the automatic attack when the virtual character is in the automatic attack state. When the function module for controlling the sound effect playback receives the attack instruction corresponding to the above-mentioned first triggering operation, it plays the musical sound effects corresponding to the attack sound effect.

[0050] In some embodiments, if there is only one musical sound effect in the current sound effect set, when the user interface receives the first triggering operation and needs to play the musical sound effect, the musical sound effect in the current sound effect set can be directly played simultaneously with the music segment to achieve the response to the first triggering operation. In some embodiments, considering that repeated triggering and playing of a musical sound effect will make the playback effect of the musical sound effect appear monotonous, in order to enrich the playback effect of the musical sound effect, two or more musical sound effects can be set in the current sound effect set; when the user interface receives the first triggering operation and needs to play the musical sound effect, a musical sound effect can be selected from the current sound effect set for playback. In some embodiments, a musical sound effect can be randomly selected from the multiple musical sound effects included in the current sound effect set for playback. The randomly selected musical sound effect can be the same as or different from the musical sound effect played last time, which can enrich the playback effect of the musical sound effect. In some embodiments, a musical sound effect in the current sound effect set can also be selected based on a preset order. For example, if the first musical sound effect, the second musical sound effect, and the third musical sound effect arranged in a preset order are included in the current sound effect set, if the first musical sound effect was played in response to the first triggering operation last time, then the second musical sound effect can be played in the preset order in response to the first triggering operation this time, which can also enrich the playback effect of the musical sound effect.

[0051] In some embodiments, when dividing musical sound effects to generate a sound effect set, multiple musical sound effects corresponding to the same musical property can be divided into the same sound effect set, that is, the sound effect set can include multiple musical sound effects corresponding to the same musical property. Among them, the musical property can be a parameter reflecting the music playing effect, such as pitch, timbre, melody, rhythm, chord, etc. In some embodiments, if two musical segments with the same or different musical properties can produce a harmonious playing effect when played simultaneously, then it can be said that these two same or different musical properties match. In some embodiments, when the sound effect set includes musical sound effects corresponding to the same musical property, one sound effect set can correspond to one musical property. The musical property corresponding to the current sound effect set can be selected according to the musical property of the currently played musical segment, such that the musical property corresponding to the current sound effect set matches the musical property of the currently played musical segment, so that the musical sound effects in the sound effect set can produce a harmonious playing effect when played simultaneously with the musical segment.

[0052] In some embodiments, these musical sound effects can be marked and classified according to pitch. Pitch is one of the basic properties of sound, used to reflect the high or low of the tone. Musical sound effects with different pitches have different high or low tones. In some embodiments, a musical sound effect can be composed of a single note, and the pitch corresponding to the note can be used as the pitch corresponding to the musical sound effect. In some embodiments, a musical sound effect can also be composed of multiple notes, and the pitch corresponding to the most prominent note during playback can be selected from the multiple notes as the pitch corresponding to the musical sound effect. In some embodiments, each musical sound effect can correspond to one pitch, and the pitches of different musical sound effects can be the same or different.

[0053] In some embodiments, the musical attribute may be a chord formed by different pitches according to a fixed arrangement. For example, it may be a C chord, a Gm chord, an Fm chord, etc. Among them, the C chord is a major chord formed by stacking the three notes C, E, and G. The Gm chord is a minor chord formed by stacking the three notes G, bB, and D. The Fm chord is a minor chord formed by stacking the three notes F, bA, and C. In some embodiments, when the musical attribute is a chord formed by different pitches according to a fixed arrangement, the pitch corresponding to the musical sound effect of the same musical attribute is the pitch that forms the chord. For example, for the musical sound effect corresponding to the C chord as a musical attribute, the pitch corresponding to it is one of the three pitches C, E, and G. Another example is that for the musical sound effect corresponding to the Gm chord as a musical attribute, the pitch corresponding to it is one of the three pitches G, bB, and D. That is, the musical attribute corresponding to the sound effect set can be a chord formed by the pitches corresponding to different musical sound effects according to a fixed arrangement. In some embodiments, the chord types corresponding to different sound effect sets may be different, but the pitches corresponding to different chord types may overlap. For example, in the C chord and the Fm chord, there is the pitch C. The musical sound effect corresponding to the pitch C can appear in the sound effect set corresponding to the C chord and the sound effect set corresponding to the Fm chord at the same time. That is, a musical sound effect can be included in two or more sound effect sets at the same time.

[0054] Figure 4 is a schematic diagram of the classification structure of a sound effect set shown in some embodiments of this specification. In some embodiments, as Figure 4 shown in the schematic diagram of the classification structure 400, it may include a first sound effect set 410 and a second sound effect set 420. Among them, the musical attribute corresponding to the first sound effect set 410 is the C chord, and the musical attribute corresponding to the second sound effect set 420 is the Gm chord. The name "C" of the C chord and the name "Gm" of the Gm chord can be used as the identifiers of the first sound effect set 410 and the second sound effect set 420 respectively. In some embodiments, as Figure 4 shown, the first sound effect set 410 includes a first musical sound effect 410a, a second musical sound effect 410b, and a third musical sound effect 410c. Among them, the pitch corresponding to the first musical sound effect 410a is "C", the pitch corresponding to the second musical sound effect 410b is "E", and the pitch corresponding to the third musical sound effect 410c is "G", all of which belong to the pitches under the C chord. The pitches "C", "E", and "G" can be used as the identifiers of the first musical sound effect 410a, the second musical sound effect 410b, and the third musical sound effect 410c respectively. In some embodiments, as Figure 4As shown, the second sound effect set 420 includes a fourth musical tone sound effect 420a, a fifth musical tone sound effect 420b, a sixth musical tone sound effect 420c, and a seventh musical tone sound effect 420d. Among them, the pitch corresponding to both the fourth musical tone sound effect 420a and the fifth musical tone sound effect 420b is "G", and the signs of the fourth musical tone sound effect 420a and the fifth musical tone sound effect 420b can be "G1" and "G2" respectively, which are the pitch "G" plus the serial number. The pitch corresponding to both the sixth musical tone sound effect 420c and the seventh musical tone sound effect 420d is "bB", and the signs of the sixth musical tone sound effect 420c and the seventh musical tone sound effect 420d can be "bB1" and "bB2" respectively, which are the pitch "bB" plus the serial number. Both the pitch "G" and the pitch "bB" belong to the pitches under the Gm chord.

[0055] In some embodiments, a chord can provide harmonic support for a music segment. For a pre-configured music segment, a production staff can obtain the chords required for each part of the music segment and select a sound effect set with the same chord music attribute as the current sound effect set according to the chords required for each part of the music segment. Figure 5 It is a schematic diagram of the setting of another sound effect switching identifier shown according to some embodiments of this specification. In some embodiments, as Figure 5 shown, a first sound effect switching identifier 521 and a second sound effect switching identifier 522 are set in the music segment 510. For the music part 510a in the music segment 510 located between the first sound effect switching identifier 521 and the second sound effect switching identifier 522, if the required chord is the C chord, then the production staff can correspond the first sound effect switching identifier 521 with the sound effect set having the C chord music attribute and use the name "C" of the C chord as the sign of the first sound effect switching identifier 521. When the music part 510a of the music segment 510 is being played, since the musical tone sound effects included in the current sound effect set all belong to the same C chord, any one of the musical tone sound effects can form a harmonious interval when played simultaneously with the currently played music part 510a. A harmonious interval refers to the pitch relationship when two notes sound simultaneously that can produce a harmonious effect. Common harmonious intervals can include major third, minor third, perfect fifth, etc., which will not be elaborated here.

[0056] In some embodiments, musical sound effects can also be marked and classified according to timbre styles. Timbre is one of the basic attributes of sound, used to reflect the distinctive characteristics of different sound waveforms. Since different object vibrations have different characteristics, different objects have different timbre styles when producing sound, and the musical sound effects corresponding to different objects also have different timbre styles. In some embodiments, musical sound effects with the same timbre style can be grouped into a timbre group in the sound effect set, that is, the sound effect set includes multiple timbre groups with different timbre styles, and each timbre group includes at least one musical sound effect. In some embodiments, in a virtual game scenario, the musical sound effect can be the attack sound effect generated when a weapon or equipment attacks. In some embodiments, the weapon or equipment can include a clarinet music weapon, a maraca music weapon, a war drum music weapon, a resonance egg music weapon, etc. Each type of music weapon has a corresponding specific timbre style. For example, the timbre style of the clarinet music weapon is the first timbre style corresponding to clarinet performance, and the timbre style of the war drum music weapon is the second timbre style corresponding to war drum beating, and so on. The musical sound effects corresponding to the clarinet music weapon can be placed in the timbre group corresponding to the first timbre style, and the musical sound effects corresponding to the war drum music weapon can be placed in the timbre group corresponding to the second timbre style to achieve marking and classification of musical sound effects according to timbre styles. Figure 6 is a schematic diagram of another classification structure of a sound effect set shown in some embodiments of this specification. In some embodiments, as Figure 6 shown, the sound effect set 610 can include multiple timbre groups 620, and each timbre group 620 contains at least one musical sound effect 621. In some embodiments, as Figure 6 shown, the musical sound effects 621a and 621b with the same pitch "F" can be divided into different timbre groups 620 according to the different sound-producing objects corresponding to the musical sound effects. Figure 7 is a schematic diagram of another classification structure of a sound effect set shown in some embodiments of this specification. In some embodiments, as Figure 7 shown, based on the classification of the sound effect set 710 according to chord music attributes, multiple timbre groups 720 can be further divided according to timbre styles. Each timbre group 720 contains at least one musical sound effect 721, and the pitch corresponding to each musical sound effect 721 belongs to the C chord music attribute corresponding to the sound effect set 710.

[0057] In some embodiments, when the sound effect set includes multiple timbre groups, when responding to a first trigger operation and playing the musical tone sound effects in the current sound effect set while playing a music segment, the target timbre style corresponding to the first trigger operation can be determined first based on the first trigger operation. In some embodiments, when the first trigger operation is that a user controls a virtual musical instrument to perform, the timbre style corresponding to the musical tone sound effect can be determined according to the type of musical instrument corresponding to the virtual musical instrument controlled by the user, and this timbre style can be used as the target timbre style corresponding to the first trigger operation. For example, when the user controls a virtual flute musical instrument to perform, the timbre style corresponding to the virtual flute musical instrument can be used as the target timbre style. In some embodiments, after determining the target timbre style, while playing the music segment, the timbre group corresponding to the target timbre style can be determined from the current sound effect set, and a musical tone sound effect in the timbre group with the target timbre style can be randomly selected or selected based on a preset order for playback, so as to achieve a harmonious playback effect between the simultaneously played musical tone sound effects and the music segment.

[0058] In some embodiments, when the audio playback method provided in the present disclosure is applied to a virtual game scenario, the musical tone sound effect can be set as the attack sound effect of a weapon. Since the attack sound effects generated by different types of weapons have different timbres when attacking, in the above embodiments, the timbre style can be classified based on the weapon category, and each timbre group includes musical tone sound effects corresponding to the timbre style of the same type of weapon. For example, in a virtual game scenario, the attack sound effect of a sword-type weapon is different from that of a stick-type weapon. The musical tone sound effects corresponding to the sword-type weapon can be divided into the same timbre group, and the musical tone sound effects corresponding to the stick-type weapon can be divided into another timbre group. In some embodiments, when the sound effect set includes multiple timbre groups, when responding to a first trigger operation and playing the musical tone sound effects in the current sound effect set while playing a music segment, the target weapon category corresponding to the first trigger operation can be determined first based on the first trigger operation. For example, according to the first trigger operation, the weapon category corresponding to the weapon held by the game character controlled by the user at the time of the first trigger operation can be determined as the target weapon category corresponding to the first trigger operation, etc.; then the target timbre style corresponding to the target weapon category can be determined based on the target weapon category.

[0059] In some further embodiments, when classifying the timbre style based on the weapon category, the weapon category may include musical weapons. In some embodiments, musical weapons, also known as instrument weapons, are special weapons that use conventional musical instruments as weapons. For such weapons, special sound effects that conform to the weapon's positioning need to be given in the trigger sound design. For example, when the above musical weapon or instrument weapon attacks, a note or a combination of a group of notes played by the instrument corresponding to the musical weapon or instrument weapon can be played. For example, when the clarinet cannon weapon triggers an attack, the corresponding attack sound effect is a short segment of musical sound played by the clarinet instrument; another example is that when the harp cannon weapon triggers an attack, the corresponding attack sound effect is a short segment of musical sound played by the harp instrument, so that the triggered attack sound effect is more in line with the self - design content of the musical weapon or instrument weapon, improving the user experience.

[0060] In some embodiments, a music segment may contain an integer number of musical measures, where a musical measure is a unit used to represent the musical beat in a music segment. In some embodiments, the total playing time of a music segment can be divided into several equal basic units, and each unit is called a "beat" or "one beat". The time length corresponding to each beat is a relative time concept. For example, when the specified speed of the music segment is 60 beats per minute, each beat occupies one second, and a half - beat occupies half a second; another example is that when the specified speed of the music segment is 120 beats per minute, each beat occupies half a second, and a half - beat occupies a quarter of a second, and so on. In some embodiments, beats can be divided into strong beats and weak beats according to the rhythm strength. For a music segment, it is composed of strong beats and weak beats distributed according to a certain rule. For example, it can be cyclically composed according to the rule of "strong beat, weak beat", or it can be cyclically composed according to the rule of "strong beat, weak beat, weak beat". The combination rule between the above - mentioned strong beats and weak beats is called the musical rhythm. A musical rhythm can contain only one strong beat, and the strong beat can be set to always be located at the starting position of the combination rule. In some embodiments, a musical measure in a music segment can specifically represent the part of the music segment from the appearance of one strong beat to the appearance of the next strong beat. In some embodiments, the musical measures in a music segment can be divided according to the measure lines. Figure 8 shows a schematic diagram of the division of a musical measure according to some embodiments of this specification, as Figure 8 shown, the music segment 800 is divided into several musical measures 820 by the measure lines 810.

[0061] In some embodiments, the sound effect switching identifier can be located at the starting position of the musical measure. As Figure 8As shown, the first sound effect switching identifier 831 can be set at the starting position of the music measure 820a, the second sound effect switching identifier 832 can be set at the starting position of the music measure 820b, and the third sound effect switching identifier 833 can be set at the starting position of the music measure 820e. As Figure 8 shown, there is one music measure between the first sound effect switching identifier 831 and the second sound effect switching identifier 832, and there are three music measures between the second sound effect switching identifier 832 and the third sound effect switching identifier 833. In some embodiments, the first sound effect switching identifier 831 can correspond to a first sound effect set, the second sound effect switching identifier 832 can correspond to a second sound effect set, and the third sound effect switching identifier 833 can correspond to a third sound effect set; then for the Figure 8 music segment 800 shown, the first sound effect set is the current sound effect set corresponding to the music measure 820a therein, the second sound effect set is the current sound effect set corresponding to the music measures 820b, 820c, and 820d therein, and the third sound effect set is the current sound effect set corresponding to the music measure 820e therein. In some embodiments, based on the above method of setting the sound effect switching identifier, each current sound effect set can correspond to one or a continuous integer number of music measures in the music segment. Since the music segment content in a single music measure or a continuous multiple music measures often has the same music attributes such as harmonic structure, the corresponding current sound effect set can be determined according to these music attributes so that the simultaneous playback of the musical sound effects in the current sound effect set and the music segment produces a harmonious playback effect.

[0062] In some embodiments, in addition to being associated with the triggering of musical sound effects, the first triggering operation may also be associated with a change in the displayed content. For example, in a virtual game scenario, when a player controls a game character to use a weapon to attack through the first triggering operation, not only will the musical sound effect corresponding to the weapon attack be triggered, but also an animation clip corresponding to the weapon attack will be displayed on the user interface. That is, in the above embodiments, in response to the first triggering operation, an animation clip corresponding to the first triggering operation may also be displayed on the user interface. In some embodiments, in order to ensure that the animation clip displayed on the user interface can match the triggered musical sound effect and avoid the situation of out-of-sync audio and video affecting the user experience, at least one sound effect triggering identifier may be set in the played animation clip, and the sound effect triggering identifier can be used to trigger the musical sound effect. In some embodiments, when passing through the sound effect triggering identifier during the playback of the animation clip, the musical sound effect in the current sound effect set may be played while playing the music clip. For example, in a scenario where a user controls a virtual character to perform musical instrument playing, the first triggering operation input by the user may represent "controlling the virtual character to play the piano". The above first triggering operation corresponds to an animation clip of the virtual character playing the piano, which may specifically include several action combinations such as the virtual character raising the hand, reaching out to pluck the strings, and resetting after plucking the strings. Playing the musical sound effect corresponding to the piano sound before or after the virtual character reaches out to pluck the strings is displayed on the user interface will cause the situation of out-of-sync audio and video. When the animation of the virtual character reaching out to pluck the strings is displayed on the user interface, it is exactly the ideal time to trigger and play the corresponding musical sound effect. Therefore, the sound effect triggering identifier can be set at the time point when the virtual character reaches out to pluck the strings, avoiding the situation that the musical sound effect is played earlier than the displayed content of the animation and thus affecting the user experience due to out-of-sync audio and video.

[0063] In some embodiments, when the audio playback method provided in the present disclosure is applied to a virtual game scenario, the music clip may be the background music corresponding to the game screen, and the game screen may be displayed through the user interface. In some embodiments, the played music clip may change accordingly as the content presented on the game screen changes. In the above embodiments, the first triggering operation may include triggering a weapon or equipment in the game screen, and the weapon or equipment has a timbre style corresponding to its weapon category. For example, the first triggering operation may be to control the weapon or equipment in the game screen to attack through an attack virtual button presented on the user interface to trigger the musical sound effect corresponding to the weapon attack. In some embodiments, the weapons or equipment in the virtual game scenario may include a clarinet music weapon, a maraca music weapon, a war drum music weapon, a resonance egg music weapon, etc. Each type of music weapon has a corresponding specific timbre style. For example, the timbre style of the clarinet music weapon is the timbre style corresponding to clarinet playing, and the timbre style of the war drum music weapon is the timbre style corresponding to war drum beating, and so on.

[0064] Some embodiments of this specification also provide an audio playback system. Figure 9 is an exemplary block diagram of an audio playback system shown in some embodiments of this specification. In some embodiments, as Figure 9 shown, the system 900 may include a music module 910 and a sound effect switching module 920.

[0065] In some embodiments, the music module 910 is used to play music segments, and at least one sound effect switching identifier is set in the music segments. The sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect.

[0066] In some embodiments, the sound effect switching module 920 is used to set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when passing through the sound effect switching identifier during the playback of the music segment.

[0067] In some embodiments, the music module 910 is further used to respond to a first trigger operation of the user interface and play the musical sound effects in the current sound effect set while playing the music segment.

[0068] Some embodiments of this specification also provide an audio playback system applied to a virtual game scenario. Figure 10 is an exemplary block diagram of another audio playback system shown in some embodiments of this specification. In some embodiments, as Figure 10 shown, the system 1000 includes a game engine 1010 and an audio engine 1020. In some embodiments, in the production scenario of the virtual game, the game engine 1010 is a comprehensive software framework for developing and running video games, providing a series of tools, interfaces, and functional modules, aiming to simplify the game development process. Its core role is to enable developers to focus on the implementation of game logic and creative content by integrating multiple underlying technologies (such as rendering, physical simulation, audio processing, animation, scripting system, network communication, etc.); the audio engine 1020 is a subsystem specifically for handling audio-related tasks, which can exist independently or as a sub-engine of the game engine. In some embodiments, the operations for corresponding sound effect switching and sound effect playback according to the played music segment are all integrated in the audio engine 1020 for implementation, which can avoid the resource loading and unloading delay when the game engine 1010 performs programming control on the sound effect switching and more accurately implement the switching control of the sound effects. In some embodiments, the audio engine 1020 may be the Wwise audio engine, which is used for sound effect and music design in games and interactive media and allows production staff to intuitively create and manage audio without coding.

[0069] In some embodiments, in the running scenario of the virtual game, the game engine 1010 may include a user interface 1011 provided to the player and an input module 1012 for receiving the player's game operation instructions. In some embodiments, the user interface 1011 may be used to display game screens; the input module 1012 may be used to obtain a first trigger operation related to sound effect playback provided by the player through a terminal device or other human-machine interface devices, and the input module 1012 may also be used to obtain other operation instructions related to the game screen provided by the player through the terminal device or other human-machine interface devices; the audio engine 1020 may play corresponding game background music segments according to the game screens displayed by the user interface 1011, and play corresponding musical sound effects according to the first trigger operation received by the input module 1012. In some embodiments, the audio engine 1020 may include a music module 1021 and a sound effect switching module 1022, where the music module 1021 is used to play music segments corresponding to the game screens, and at least one sound effect switching identifier is set in the music segments, and the sound effect switching identifier corresponds to a sound effect set including at least one musical sound effect; the sound effect switching module 1022 is used to set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when passing through the sound effect switching identifier during the playback of the music segments. In some embodiments, the music module 1021 may also respond to the first trigger operation and play the musical sound effects in the current sound effect set while playing the music segments.

[0070] In some embodiments, the user interface 1011 may also be used to respond to the first trigger operation and display an animation segment corresponding to the first trigger operation while displaying the game screen, and at least one sound effect trigger identifier is set in the animation segment. In some embodiments, when passing through the sound effect trigger identifier during the playback of the animation segment, the music module 1022 plays the musical sound effects in the current sound effect set while playing the music segments.

[0071] For more information about each module, reference can be made to Figure 2 the relevant descriptions in the figures, which will not be elaborated here. It should be understood that Figure 9 and Figure 10The system and its modules shown can be implemented in various ways. For example, in some embodiments, the system and its modules can be implemented through hardware, software, or a combination of software and hardware. Among them, the hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. Those skilled in the art can understand that the above methods and systems can be implemented using computer-executable instructions and / or control codes included in a processor. For example, such codes are provided in carrier media such as magnetic disks, CDs, or DVD-ROMs, or in the memories of programmable devices. The systems and modules in this specification can be implemented not only by hardware circuits such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips and transistors, or programmable hardware devices such as field programmable gate arrays and programmable logic devices, but also by software executed by various types of processors, or by a combination of the above hardware circuits and software (e.g., firmware).

[0072] It should be noted that the above description of the system and its modules is only for convenience of description and does not limit this specification to the scope of the examples given. It can be understood that for those skilled in the art, after understanding the principle of the system, they may, without departing from this principle, make any combination of the various modules to form a subsystem connected to other modules. Or split some modules to obtain more modules or multiple units under that module. Such deformations are all within the scope disclosed in this specification.

[0073] In some embodiments provided in this specification, a computer program product is also provided, including computer instructions or a computer program. When at least part of the computer instructions or the computer program is executed by a processor, it can implement the audio playback method provided in the foregoing embodiments of this specification.

[0074] In some embodiments, the above processor can be a combination of one or more of the following processors: central processing unit (CPU), application specific integrated circuit (ASIC), application specific instruction set processor (ASIP), graphics processing unit (GPU), physics processing unit (PPU), digital signal processor (DSP), field programmable gate array (FPGA), programmable logic device (PLD), programmable logic controller (PLC), reduced instruction set computer (RISC), microprocessor.

[0075] The beneficial effects that may be brought about by the embodiments of this specification include, but are not limited to: (1) When it is necessary to play musical sound effects, the musical sound effects in the matching sound effect set can be selected according to the playback situation of the music segment (including the part of the music segment currently being played), so that the simultaneously played musical sound effects and the music segment produce a harmonious playback effect, enhancing the user experience. (2) When dividing the sound effect set, it can be divided according to the musical attributes such as pitch and timbre of the musical sound effects in the sound effect set, so that the musical attributes corresponding to the current sound effect set match the musical attributes of the currently played music segment, ensuring a harmonious playback effect. (3) The sound effect switching identifier can be set according to the music measure, making the effect of sound effect switching more accurate. (4) When playing musical sound effects in response to the first trigger operation, the playback can be based on the sound effect trigger identifier in the animation segment corresponding to the first trigger operation, ensuring the audio-visual synchronization during the sound effect playback process. (5) Integrating the operations of corresponding sound effect switching and sound effect playback according to the played music segment into the audio engine can avoid the loading and unloading delay of resources when the game engine programmatically controls the sound effect switching, and more quickly and accurately achieve the switching control of the sound effects. It should be noted that the beneficial effects that may be produced by different embodiments are different. In different embodiments, the beneficial effects that may be produced can be any one or several combinations of the above, or any other beneficial effects that may be obtained.

[0076] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only an example and does not constitute a limitation to this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements, and corrections to this specification. Such modifications, improvements, and corrections are taught in this specification, so such modifications, improvements, and corrections still fall within the spirit and scope of the exemplary embodiments of this specification.

Claims

1. An audio playback method, characterized in that: The method comprises: Playing a music clip, wherein the music clip is provided with at least one sound effect switching mark, and the sound effect switching mark corresponds to a sound effect set including at least one musical sound effect; When the music clip passes through the sound effect switching mark during playback, setting the sound effect set corresponding to the sound effect switching mark as the current sound effect set; In response to the first trigger operation, the musical sound effects in the current sound effect set are played while the music clip is played.

2. The method according to claim 1, characterized in that The step of playing the musical sound effects in the current sound effect set while playing the music clip comprises: While playing the music clip, a musical sound effect in the current sound effect set is selected randomly or based on a preset order for playing.

3. The method according to claim 1, characterized in that The sound effect set includes a plurality of musical sound effects corresponding to the same musical attribute; The music attribute corresponding to the current sound effect set matches the music attribute of the currently playing music clip.

4. The method according to claim 3, characterized in that: The music attribute is a chord formed by pitches corresponding to different musical sound effects according to a fixed arrangement; Each of the musical sound effects corresponding to the same musical attribute contained in the current sound effect set forms a harmonious interval with the music clip currently being played.

5. The method according to claim 3 or 4, characterized in that: The sound effect set includes a plurality of timbre groups with different timbre styles, each of which includes at least one musical sound effect: In response to the first trigger operation, playing the musical sound effects in the current sound effect set while playing the music clip comprises: Determining a target timbre style corresponding to the first trigger operation based on the first trigger operation; While playing the music clip, a musical sound effect in a timbre group having the target timbre style is selected randomly or based on a preset order from the current sound effect set for playing.

6. The method according to claim 5, characterized in that The sound styles are classified based on weapon categories; Each of the timbre groups includes musical sound effects corresponding to the same weapon timbre style; The determining of a target timbre style corresponding to the first trigger operation based on the first trigger operation includes: Determining a target weapon category corresponding to the first trigger operation based on the first trigger operation; The target timbre style corresponding to the target weapon category is determined based on the target weapon category.

7. The method according to claim 6, characterized in that Said weapon categories include musical weapons.

8. The method according to claim 1, characterized in that In response to the first trigger operation, playing the musical sound effect in the current sound effect set while playing the music clip comprises: In response to the first trigger operation, displaying a cartoon segment corresponding to the first trigger operation on a user interface, wherein at least one sound effect trigger mark is set in the cartoon segment; When the animation segment passes the sound effect trigger mark during the playing process, the musical sound effect in the current sound effect set is played while playing the music segment.

9. The method according to claim 1, characterized in that: The music clip includes an integer number of music measures, and the sound effect switching mark is located at the starting position of the music measure.

10. The method according to claim 1 or 6, characterized in that: The music clip corresponds to the background music of the game screen, and the game screen is displayed through the user interface; The first trigger operation includes triggering the weapons and equipment in the game screen; wherein the weapons and equipment have a tone style corresponding to their weapon category.

11. An audio playback system, characterized in that: The system comprises: A music module, used for playing a music clip, wherein the music clip is provided with at least one sound effect switching mark, and the sound effect switching mark corresponds to a sound effect set including at least one musical sound effect; A sound effect switching module, configured to set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when the sound effect switching identifier is passed during the playing of the music clip; In response to the first trigger operation, the music module plays the musical sound effects in the current sound effect set while playing the music clip.

12. An audio playback system, characterized in that: The system comprises: The game engine comprises a user interface and an input module; the user interface is used to display a game screen; the input module is used to obtain a first trigger operation; Audio engine, including music module and sound effect switching module; The music module is used to play a music clip corresponding to the game screen, wherein the music clip is provided with at least one sound effect switching mark, and the sound effect switching mark corresponds to a sound effect set including at least one musical sound effect; The sound effect switching module is used to set the sound effect set corresponding to the sound effect switching identifier as the current sound effect set when the sound effect switching identifier is passed during the playing of the music clip; In response to the first trigger operation, the music module plays the musical sound effects in the current sound effect set while playing the music clip.

13. The audio playback system according to claim 12, characterized in that: The user interface is further configured to respond to the first trigger operation by displaying an animation clip corresponding to the first trigger operation while displaying the game screen, wherein at least one sound effect trigger mark is provided in the animation clip; When the animation segment passes the sound effect trigger mark during the playing process, the music module plays the musical sound effect in the current sound effect set while playing the music segment.

14. A computer program product, comprising computer instructions or a computer program, wherein when at least part of the computer instructions or the computer program is executed by a processor, the audio playback method according to any one of claims 1 to 10 can be implemented.

Images