Audio resource processing method and device, storage medium, equipment and program product

Abstract

The invention discloses an audio resource processing method and device, a storage medium, equipment and a program product, which are applied to an audio resource processing scene. The method comprises the following steps: in response to an audio playing request of a target audio event, searching whether a target asset file of the target audio event exists in an audio event pool or not; if not, asynchronously loading a target asset file of the target audio event, and loading a target audio resource referenced in the target asset file from a memory of an application development engine, the memory being used for storing asset files of a plurality of audio events synchronized from an audio engine to the application development engine, each asset file at least comprising an audio resource, the asset file is generated based on an event packaging mechanism; registering a target audio event by calling an application program interface of an audio engine to play a target audio resource; and when the target audio event is finished, the target audio resource is unloaded, so that the audio memory occupation is reduced, and the management efficiency and utilization efficiency of the audio resource are improved.

Description

Technical Field

[0001] This application relates to the field of information processing technology, specifically to an audio resource processing method, apparatus, storage medium, device, and program product. Background Technology

[0002] In the current field of audio resource management, especially in the development of large-scale open-world games and mobile games, effective management and optimization of audio resources are key factors in improving game performance and user experience. For realistic open-world games, the amount of audio used is enormous, which can easily lead to difficulties in managing and categorizing audio resources, resulting in huge maintenance costs. Furthermore, mobile games prioritize memory; if audio memory usage cannot be reduced, it will cause performance issues. Even if audio memory usage is reduced, the categorization of audio files can lead to redundant resources loaded into memory.

[0003] While traditional bank-based audio packaging technology meets the needs of audio resource management to some extent, it suffers from problems such as large memory redundancy and high maintenance costs. This packaging method typically packages the entire audio file or a group of audio files into a large soundbank file. As a result, during game operation, all audio resources are loaded into memory regardless of whether they are needed, thus consuming a large amount of memory resources, especially when the audio resources are large.

[0004] Furthermore, Bank-based audio packaging technology requires developers to manually organize and manage Bank files, increasing workload and the possibility of errors. For mobile games, memory resources are particularly precious; excessive audio memory usage can directly impact game smoothness and performance, and even cause application crashes.

[0005] Therefore, it is necessary to develop a more efficient and flexible audio resource processing technology. Summary of the Invention

[0006] This application provides an audio resource processing method, apparatus, storage medium, device, and program product. By introducing an event packaging mechanism and the concept of an audio event pool, it realizes on-demand loading and unloading of audio resources, reduces audio memory usage, and improves the management and utilization efficiency of audio resources.

[0007] On one hand, embodiments of this application provide an audio resource processing method, the method comprising:

[0008] In response to an audio playback request for a target audio event, check if the target asset file for the target audio event exists in the audio event pool;

[0009] If the target asset file of the target audio event does not exist in the audio event pool, the target asset file of the target audio event is loaded asynchronously, and the target audio resources referenced in the target asset file are loaded from the memory of the application development engine. The memory is used to store asset files of multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources. The asset file is generated based on the event packaging mechanism.

[0010] Register the target audio event by calling the application programming interface of the audio engine to play the target audio resource;

[0011] When the target audio event ends, the target audio resource is unloaded.

[0012] On the other hand, embodiments of this application provide an audio resource processing apparatus, the apparatus comprising:

[0013] The first processing unit is used to respond to an audio playback request of a target audio event by checking whether the target asset file of the target audio event exists in the audio event pool.

[0014] The second processing unit is configured to asynchronously load the target asset file of the target audio event if the target asset file of the target audio event does not exist in the audio event pool, and load the target audio resources referenced in the target asset file from the memory of the application development engine. The memory is used to store asset files of multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources. The asset file is generated based on the event packaging mechanism.

[0015] The third processing unit is used to register the target audio event by calling the application programming interface of the audio engine in order to play the target audio resource;

[0016] An unloading unit is used to unload the target audio resource when the target audio event ends.

[0017] In some embodiments, the device is further configured to:

[0018] Display the resource management interface of the application development engine;

[0019] In response to an asset synchronization command entered on the resource management interface, the asset files of multiple audio events in the audio engine are synchronized to the memory of the application development engine.

[0020] In some embodiments, before synchronizing asset files of multiple audio events in the audio engine to the memory of the application development engine, the apparatus is further configured to:

[0021] When it is detected that a media memory region for storing the asset file has been allocated in the memory and the media memory region is not empty, the corresponding content release function is selected according to the configuration options to release the media memory region;

[0022] When the media memory region is released, the metadata in the data block is cleared;

[0023] Update the memory usage statistics.

[0024] In some embodiments, the device is further configured to:

[0025] In response to an export command for the first asset file being entered on the resource management interface, the first asset file is exported from the memory; or

[0026] In response to a first import command for the second asset file being entered on the resource management interface, the second asset file is imported into the memory; or

[0027] In response to the input of a second import command for the first audio resource on the resource management interface, the first audio resource is imported into the memory, and a third asset file corresponding to the first audio resource is generated.

[0028] In some embodiments, the step of generating the asset file based on an event packaging mechanism includes:

[0029] The audio engine divides audio resource data into multiple audio events, and each audio event includes at least one audio resource segment and the metadata corresponding to the audio resource segment.

[0030] The audio engine generates asset files for each audio event based on an event packaging mechanism, and the file format of the asset files is a format that the application development engine can recognize.

[0031] In some embodiments, the device is further configured to:

[0032] A main thread, an asynchronous loading thread, and an audio processing thread are created respectively. A communication connection is established between the main thread and the asynchronous loading thread, and a communication connection is established between the main thread and the audio processing thread.

[0033] The main thread is used to respond to the audio playback request of the target audio event, check whether the target asset file of the target audio event exists in the audio event pool, and unload the target audio resource by calling the playback end callback function when the target audio event ends;

[0034] The asynchronous loading thread is used to asynchronously load the target asset file of the target audio event and load the target audio resources referenced in the target asset file from the memory of the application development engine;

[0035] The audio processing thread is used to register the target audio event by calling the application interface of the audio engine through the playback start callback function, so as to play the target audio resource.

[0036] In some embodiments, the playback interface for playing the target audio resource is one of preset playback interfaces, the preset playback interface including:

[0037] The first playback interface is used to play two-dimensional sound effects at the origin position in virtual three-dimensional space;

[0038] The second playback interface is used to play three-dimensional sound effects in the virtual three-dimensional space, following the overall position changes of the virtual character.

[0039] The third playback interface is used to play three-dimensional sound effects in the virtual three-dimensional space, following the local positional changes of the target part of the virtual character.

[0040] The fourth playback interface is used to play 3D sound effects at specified coordinates in virtual 3D space.

[0041] In some embodiments, the device is further configured to:

[0042] In response to an audio pause request from the target audio resource, if the target audio resource is in playback mode, playback of the target audio resource is paused by calling the application programming interface of the audio engine; or

[0043] In response to an audio pause request from the target audio resource, if the target audio resource is in a loading state, the loading of the target audio resource continues until loading is complete, and the playback of the target audio resource is paused by calling the application programming interface of the audio engine according to the audio pause request.

[0044] In some embodiments, the device is further configured to:

[0045] The application development engine displays the debugging and management interface corresponding to the target application.

[0046] In response to triggering the audio debugging switch to be turned on on the debugging management interface, the audio debugging window of the target application is displayed.

[0047] In some embodiments, the device is further configured to:

[0048] The application development engine displays the animation development interface corresponding to the target application. This interface is configured with an animation notification status interface, which is used to trigger corresponding audio events under the target state of the animation within the target application; or...

[0049] The application development engine displays the skill editor corresponding to the target application. The skill editor is configured with a skill playback audio task, which is used to play the audio resource corresponding to the target skill when the target skill is executed.

[0050] In some embodiments, the device is further configured to:

[0051] Establish a path mapping relationship between the event names and simplified paths of each audio event;

[0052] Before playing the target audio resource, the target audio resource is loaded through the path mapping relationship.

[0053] On the other hand, an embodiment of this application provides a computer-readable storage medium storing a computer program adapted for loading by a processor to execute the audio resource processing method as described in any of the above embodiments.

[0054] On the other hand, an embodiment of this application provides a computer device, which includes a processor and a memory. The memory stores a computer program, and the processor executes the audio resource processing method as described in any of the above embodiments by calling the computer program stored in the memory.

[0055] On the other hand, an embodiment of this application provides a computer program product, including computer instructions, which, when executed by a processor, implement the audio resource processing method as described in any of the above embodiments.

[0056] This embodiment of the application, in response to an audio playback request for a target audio event, checks if a target asset file for the target audio event exists in the audio event pool. If the target asset file for the target audio event does not exist in the audio event pool, it asynchronously loads the target asset file and loads the target audio resources referenced in the target asset file from the application development engine's memory. The memory is used to store asset files for multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources, and the asset files are generated based on an event packaging mechanism. The target audio event is registered by calling the audio engine's application programming interface to play the target audio resources. When the target audio event ends, the target audio resources are unloaded. This embodiment of the application generates asset files containing audio resources based on an event packaging mechanism and synchronizes them to the application development engine. It also introduces the concept of an audio event pool, realizing on-demand loading and unloading of audio resources in the audio playback process, effectively reducing audio memory consumption and significantly improving the management and utilization efficiency of audio resources. Attached Figure Description

[0057] Figure 1 This is a schematic diagram illustrating an application scenario of the audio resource processing system provided in an embodiment of this application.

[0058] Figure 2 This is a first flowchart illustrating the audio resource processing method provided in an embodiment of this application.

[0059] Figure 3 This is a schematic diagram of the second process of the audio resource processing method provided in the embodiments of this application.

[0060] Figure 4 An example diagram of the first graphical user interface of the application development engine provided in this application embodiment.

[0061] Figure 5 An example diagram of the second graphical user interface of the application development engine provided in this application embodiment.

[0062] Figure 6 An example diagram of the first resource management interface of the application development engine provided in this application embodiment.

[0063] Figure 7 This is a schematic diagram of the process for exporting the first asset file provided in an embodiment of this application.

[0064] Figure 8 This is a schematic diagram illustrating the process of importing a second asset file, provided as an embodiment of this application.

[0065] Figure 9 This is a schematic diagram of the process for importing a first audio resource provided in an embodiment of this application.

[0066] Figure 10 This is a flowchart illustrating the multi-threaded playback method for target audio resources provided in an embodiment of this application.

[0067] Figure 11 An example diagram of the management and debugging interface provided in this application embodiment.

[0068] Figure 12 An example diagram of an audio debugging window provided in an embodiment of this application.

[0069] Figure 13 A schematic diagram of the animation development interface provided in this application embodiment.

[0070] Figure 14 Example diagram of the interface of the skill editor provided in this application embodiment.

[0071] Figure 15 This is a schematic diagram of the structure of the audio resource processing device provided in the embodiments of this application.

[0072] Figure 16 A schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0073] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0074] This application provides an audio resource processing method, apparatus, storage medium, device, and program product. Exemplarily, the audio resource processing method of this application can be executed by a computer device, which can be a terminal or a server. The terminal can be a smartphone, tablet, laptop, desktop computer, smart TV, smart speaker, wearable smart device, personal computer (PC), smart vehicle terminal, etc. The terminal may also include a client, which can be a game client, virtual reality application client, augmented reality application client, etc. The server can be an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and big data and artificial intelligence platforms.

[0075] The embodiments of this application can be applied to scenarios such as application development, game development, and audio resource processing.

[0076] First, some of the nouns or terms that appear in the description of the embodiments of this application are explained as follows:

[0077] Event-Based Packaging (EBP): EBP is a packaging method for organizing and managing audio resources. EBP uses fine-grained events as a basis, dividing audio resources into smaller units, each called an "Event" (corresponding to the audio event in this application). In EBP mode, a large audio resource is broken down into multiple "Events," each containing an audio segment and associated metadata. These "Events" can be organized and managed according to the needs of the game or application. Each "Event" can have its own trigger conditions, playback parameters, and audio effects, etc.

[0078] Unreal Engine (UE) is a comprehensive tool for creating and developing various types of games and virtual reality applications. It includes a suite of development tools and features, such as a scene editor, material editor, Blueprint system (a visual programming system), animation editor, physics simulation, and collision detection. Developers can use these tools to create game worlds, design characters, implement game logic, add special effects and animations, and more.

[0079] Waveworks Interactive Sound Engine (Wwise) is an advanced audio engine and toolkit for audio design and implementation in interactive media applications such as games, virtual reality, and augmented reality. Wwise offers powerful audio features, including audio mixing, sound effects processing, 3D sound spatial positioning, audio event systems, dynamic sound systems, audio editing, and real-time audio feedback.

[0080] The solutions provided in this application relate to technologies such as audio resource processing in application development scenarios, and are specifically illustrated through the following embodiments. Detailed descriptions are provided below. It should be noted that the order of description of the following embodiments is not intended to limit the priority of the embodiments.

[0081] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an application scenario of the audio resource processing system provided in this application embodiment. The system can implement an audio resource processing method. The audio resource processing system may include a first terminal 10 and a second terminal 20, and a communication connection is established between the first terminal 10 and the second terminal 20.

[0082] The first terminal 10 is equipped with an application development engine 11, which can be Unreal Engine. This engine is used to develop interactive applications, such as game applications, virtual reality applications, and augmented reality applications. The second terminal 20 is equipped with an audio engine 21. The audio resource processing method provided in this embodiment introduces an event packaging mechanism and the concept of an audio event pool, enabling on-demand loading and unloading of audio resources, reducing audio memory usage, and improving the management and utilization efficiency of audio resources. The first terminal 10 and the second terminal 20 can be independent devices, such as smartphones, tablets, laptops, desktop computers, smart TVs, smart speakers, wearable smart devices, personal computers (PCs), and smart vehicle terminals. They can also be different modules of the same device, such as the application development engine module and the audio engine module in a desktop computer.

[0083] Specifically, in response to a target audio event triggered by the current object in an interactive application developed based on the application development engine 11, the first terminal 10 generates an audio playback request for the target audio event. In response to the audio playback request, the first terminal 10 checks if a target asset file for the target audio event exists in the audio event pool of the application development engine 11. If the target asset file does not exist in the audio event pool, it asynchronously loads the target asset file and loads the target audio resources referenced in the target asset file from the memory of the application development engine 11. The memory is used to store asset files for multiple audio events synchronized from the audio engine 21 of the second terminal 20 to the application development engine 11 of the first terminal 10. Each asset file includes at least audio resources, and the asset files are generated based on an event packaging mechanism. The first terminal 10 registers the target audio event by calling the application programming interface of the audio engine 21 of the second terminal 20 to play the target audio resources through the audio engine 21 of the second terminal 20. The first terminal 10 unloads the target audio resources when the target audio event ends.

[0084] Please see Figure 2 and Figure 3 , Figure 2 and Figure 3 These are schematic flowcharts illustrating the audio resource processing method provided in the embodiments of this application. This method can be... Figure 1 The method, executed by the first terminal 10 shown, may include the following steps 110 to 140:

[0085] Step 110: In response to the audio playback request of the target audio event, check if the target asset file of the target audio event exists in the audio event pool.

[0086] Audio resources can be used in interactive applications developed based on application development engines. Interactive applications can be, for example, game applications, which can use audio resources to create game sound effects.

[0087] The target audio event corresponds to the target audio. When the target audio event is triggered, the target audio will be played. The target audio event can be triggered manually by the user or automatically by the first terminal 10. For example, in a game scene, the target audio event can be used to release a skill for a game character, and the corresponding target audio is the skill sound effect. Or, if the target audio event is used to perform a specific action (such as jumping or colliding) for a game character, the corresponding target audio is the action simulation sound effect that matches the action of the game character.

[0088] More specifically, the target audio event can be manually triggered by the user or automatically triggered by the interactive application developed based on the application development engine on the first terminal 10 during operation.

[0089] An audio event pool is a resource pool or collection used to manage audio events. Data in the audio event pool is stored in the form of asset files (.uasset). Each asset file includes at least one audio resource packaged based on an audio event. This audio resource can be in any form, such as unprocessed raw audio, processed audio segments, audio streams, etc.

[0090] In some embodiments, the asset file may also include definitions of audio events, such as the event name and triggering conditions of the audio event, or metadata of the audio event or audio resource, such as the author, creation date, and version number.

[0091] After the interactive application (target application) running on the first terminal 10 triggers the target audio event, or after the interactive application running on the terminal automatically triggers the target audio event, the first terminal 10 will generate an audio playback request corresponding to the target audio event. In response to the audio playback request of the target audio event, the first terminal 10 will search the audio event pool to see if the target asset file of the target audio event exists.

[0092] like Figure 3 As shown, in response to the user's triggering operation on the target audio event in the game application developed based on the application development engine, the first terminal 10 generates an audio playback request for the target audio event. In response to the audio playback request for the target audio event, the first terminal 10 attempts to load the target asset file of the target audio event.

[0093] Specifically, when attempting to load the target asset file for the target audio event, it checks whether the target asset file (AudioEvent.uasset) for the target audio event (AudioEvent) exists in the AudioEvent Pool.

[0094] Step 120: If the target asset file of the target audio event does not exist in the audio event pool, the target asset file of the target audio event is loaded asynchronously, and the target audio resources referenced in the target asset file are loaded from the memory of the application development engine. The memory is used to store asset files of multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources. The asset file is generated based on the event packaging mechanism.

[0095] The audio engine (Wwise) is an audio processing tool responsible for tasks such as playing, recording, editing, and synthesizing audio data.

[0096] In some implementations, the application development engine can be Unreal Engine (UE), and the audio engine can be integrated into the application development engine or operate independently of it. The audio engine provides audio processing and playback functionality for applications developed using the application development engine, and asset files (.uasset) are created in the audio engine and synchronized to the application development engine.

[0097] like Figure 3 As shown, if the target asset file of the target audio event does not exist in the audio event pool, the first terminal 10 asynchronously loads the target asset file of the target audio event and loads the target audio resource referenced in the target asset file from the memory of the application development engine in order to play the target audio resource.

[0098] If the target asset file for the target audio event exists in the audio event pool, the target audio resource in the target asset file will be played directly.

[0099] In some embodiments, the step of generating asset files based on an event packaging mechanism includes:

[0100] The audio engine divides audio resource data into multiple audio events, and each audio event includes at least one audio resource segment and the corresponding metadata of the audio resource segment;

[0101] The audio engine generates asset files for each audio event based on an event packaging mechanism. The asset files are in a format that the application development engine can recognize.

[0102] Since audio resource data is often quite large, packaging it as a single resource is not conducive to audio combination and management, nor to subsequent applications. Therefore, in this application, audio resource data is divided into multiple audio events. Each audio event corresponds to a specific event or a specific sound effect. For example, the audio segment corresponding to footsteps in the audio resource data is segmented and made into an audio event related to footsteps; or the audio segment corresponding to rain sounds in the audio resource data is segmented and made into an audio event related to rain sounds.

[0103] The metadata of an audio resource segment in an audio event is data used to describe that audio resource segment, providing additional information about the audio resource segment, such as author, title, tags, creation date, and playback parameters (such as volume, pitch, and tempo), which facilitates the organization and retrieval of audio events.

[0104] For example, asset files can be in a format that the application development engine can recognize, such as ".uasset".

[0105] In some embodiments, audio events may include standard audio events (AkAudioEvents) and extended audio events (AcAkAudioEvents). Standard audio events can be built-in audio events of the audio engine, while extended audio events can be audio events that the application development engine adds customizable to the actual development scenario. When implementing extended audio events, the application development engine should inherit and reuse as much of the inheritable code as possible from the standard audio events. For the parts that cannot be directly inherited, the code of the standard audio events should be copied and modified.

[0106] In some embodiments, to ensure synchronization between audio and video, the relevant standard audio resources should be loaded before playing the movie scene. If the standard audio resources related to the movie scene are not loaded when the audio resource playback interface is called, the application development engine will load the relevant standard audio resources synchronously.

[0107] In some embodiments, the audio engine can use story scenes in level sequences and widget animations to trigger and control audio events.

[0108] By generating asset files for each audio event through an EBP-based packaging optimization mechanism, the memory usage of audio resources in interactive applications developed using an application development engine can be reduced. For example, taking game A as an example, before applying the EBP packaging optimization mechanism, the memory usage of the audio resources for game A was 131MB; after applying the EBP packaging optimization mechanism, the memory usage of the audio resources for game A was reduced to 33MB, effectively reducing the memory consumption of the audio resources for game A.

[0109] In other embodiments, a more detailed and scientific classification and hierarchical packaging mechanism based on the SoundBank audio packaging mechanism can also be used to generate asset files for each audio event. This classification and hierarchical packaging mechanism generates asset files with a granularity close to that of the EBP model. One type of SoundBank audio packaging mechanism is designed for efficient management of the game's audio and motion components, organizing all the game's audio and motion data into different banks. A bank is a file containing game audio and motion data, media, or both.

[0110] Step 130: Register the target audio event by calling the audio engine's application programming interface to play the target audio resource.

[0111] like Figure 3 As shown, after the target audio resource is loaded, the first terminal 10 calls the audio engine's application programming interface (API) to register the target audio event, so that the target audio event can be triggered, and further, the target audio resource of the target audio event can be played.

[0112] In some embodiments, when playing a target audio resource, the event name of the target audio event is used as a parameter of the audio resource playback interface. The application development engine calls the audio resource playback interface based on this parameter to play the target audio resource.

[0113] Step 140: When the target audio event ends, unload the target audio resource.

[0114] like Figure 3 As shown, when the target audio event ends, that is, when the target audio event finishes playing or is no longer needed, the target audio resource is unloaded from the first terminal 10, thereby saving memory.

[0115] In other embodiments, the method further includes:

[0116] When the target audio event ends, the target audio resource is stored in the audio event pool.

[0117] When a target audio event ends, if it may be reused in the future, it can be returned to the audio event pool. The next time the target audio event is triggered or the target audio resource needs to be played again, the already loaded target audio event can be retrieved directly from the audio event pool without reloading the resource, thus improving efficiency.

[0118] In some embodiments, prior to step 110, the method may further include steps 150-160:

[0119] Step 150: Display the resource management interface of the application development engine;

[0120] Step 160: In response to the asset synchronization command entered on the resource management interface, synchronize the asset files of multiple audio events in the audio engine to the memory of the application development engine.

[0121] The application development engine's resource management interface is an operable interface used for resource management.

[0122] In some embodiments, in response to an asset synchronization instruction entered on the resource management interface, the asset files of multiple audio events in the audio engine are synchronized to the memory of the application development engine, including: in response to a first asset synchronization instruction entered on the first resource management interface, the historically updated asset files in the audio engine are synchronized to the application development engine.

[0123] The first asset synchronization command is a manual command to synchronize asset files. It can manually force the synchronization of asset files of historically updated audio events in the audio engine to the application development engine. Each time the first asset synchronization command is entered on the resource management interface, the asset files of multiple historically updated audio events in the audio engine are forcibly synchronized to the memory of the application development engine.

[0124] Specifically, the first terminal 10 displays the first resource management interface of the application development engine. In response to the first asset synchronization operation performed by the relevant user in the first resource management interface (such as clicking the setting option for forced synchronization of asset files), the application development engine generates a first asset synchronization instruction. In response to the first asset synchronization instruction, the application development engine forcibly synchronizes the asset files of multiple historically updated audio events in the audio engine to the memory of the application development engine.

[0125] like Figure 1The first terminal 10 shown is equipped with an application development engine 11, which can be Unreal Engine. This engine is used to develop interactive applications, such as game applications, virtual reality applications, and augmented reality applications. For example, a target application is currently being developed using the application development engine 11. During the development of the target application, configuration information for switching EBP modes can be input through the graphical user interface of the application development engine 11.

[0126] For example, in a method for switching EBP modes, in such Figure 4 On the first graphical user interface, you can set the relevant configuration information for sound, such as filling in "Wwise Sound DataFolder" as "WwiseAudioEBP", checking "Split Switch ContainerMedia", and checking "Use Event-Based Packaging".

[0127] For example, in another method of switching EBP modes, in Figure 5 On the second graphical user interface shown, you can set the "Wwise Windows installation path" for the audio engine, for example, setting it to "C:\Program Files(x86)\Audiokinetic\Wwise 2021.1.3.7665". By setting this installation path, you can change the audio design to the local Wwise directory. For example, this second graphical user interface also shows an option to "Enable Automatic Asset Synchronization".

[0128] For example, in completing Figure 4 or Figure 5 After configuring the EBP mode as shown, if the application development engine is enabled for asset synchronization, the following display can be shown: Figure 6 The first resource management interface of the application development engine is shown.

[0129] See Figure 6 , Figure 6This is an example diagram of the first resource management interface of the application development engine provided in this application embodiment. The resource management interface of the application development engine includes a Force Asset Synchronization option. In response to the user's click operation on this option, the application development engine generates a first asset synchronization instruction. In response to the first asset synchronization instruction, the application development engine manually forces the synchronization of asset files of multiple historically updated audio events in the audio engine to the memory of the application development engine.

[0130] In some embodiments, in response to an asset synchronization command entered on the resource management interface, synchronizing asset files of multiple audio events in the audio engine to the memory of the application development engine further includes:

[0131] In response to the second asset synchronization command entered in the second resource management interface, the asset files of multiple audio events that are updated in real time in the audio engine are automatically synchronized to the memory of the application development engine in real time.

[0132] The second asset synchronization command is used to automatically synchronize asset files, which can automatically synchronize the asset files of audio events that are updated in real time in the audio engine to the application development engine.

[0133] In response to the second asset synchronization command entered in the second resource management interface, the application development engine can automatically synchronize the updated asset files in the audio engine to the application development engine when the audio engine updates asset files, without needing to enter the second asset synchronization command again.

[0134] In one specific embodiment provided in this application, the second resource management interface of the application development engine (such as...) Figure 5 The second graphical user interface shown includes an option to enable automatic asset synchronization. In response to the user's click on this automatic asset synchronization option, the application development engine generates a second asset synchronization instruction. In response to this second asset synchronization instruction, the application development engine automatically synchronizes the asset files of multiple real-time updated audio events in the audio engine to the application development engine's memory in real time.

[0135] In some embodiments, before synchronizing the asset files of multiple audio events in the audio engine to the memory of the application development engine, the following steps 170-190 are also included:

[0136] Step 170: When it is detected that a media memory region for storing asset files has been allocated in memory and the media memory region is not empty, select the corresponding content release function according to the configuration options to release the media memory region.

[0137] Step 180: When releasing the media memory area, clear the metadata in the data block;

[0138] Step 190: Update memory usage statistics.

[0139] In this embodiment, a media memory region (mediaMemory) for storing asset files is allocated in the memory corresponding to the application development engine. Before synchronizing the asset files in the audio engine to the memory of the application development engine, the application development engine uses a related function (UAKMediaAssetData) to determine whether the media memory region is empty. If the media memory region is empty (FreemediaMemory), the function returns directly without performing any operation. If the media memory region is not empty, it means that the media memory region has been used by other data. Then, the content release function (such as FMemory::Free, or AKPLATFORM::FreeDevice) used to release the media memory region can be called according to different configuration options (such as AK_MEDIA_MEMORY_STAT, or AK_SUPPORT_DEVICE_MEMORY macro) to release the media memory region. The content release function can be a built-in content release function in the application development engine.

[0140] Furthermore, when releasing the media memory region, it is necessary to release the data in the data block DataChunks[0] associated with the media memory region. This data block is used to store the metadata of the asset files corresponding to the media memory region. Clearing the metadata in the data block when releasing the media memory region can prevent memory leaks.

[0141] After completing the release of the media memory area and the data cleanup of the data blocks associated with the media memory area, the first terminal 10 will update the memory usage statistics, such as the used memory information and the available memory information.

[0142] In some embodiments, the method may further include the following steps 1100-1300:

[0143] Step 1100: In response to the input of an export command for the first asset file on the resource management interface, export the first asset file from memory;

[0144] Step 1200: In response to the input of a first import command for the second asset file on the resource management interface, import the second asset file into memory;

[0145] Step 1300: In response to the input of a second import command for the first audio resource on the resource management interface, the first audio resource is imported into memory, and a third asset file corresponding to the first audio resource is generated.

[0146] In step 1100, in response to an export command for the first asset file input on the resource management interface, the first asset file is exported from memory. Specifically, this includes: in response to the first asset file export operation executed on the resource management interface, the first terminal 10 generates an export command for the first asset file; and in response to the export command for the first asset file, the first terminal 10 exports the first asset file from the memory of the application development engine. This method enables the automatic export of asset files from the application development engine. For example, the first asset file could be an asset file containing all audio events stored in the memory of the application development engine. Alternatively, the first asset file could be an asset file containing only a portion of the audio events stored in the memory of the application development engine.

[0147] Figure 7 This is a schematic diagram of the process for exporting the first asset file provided in an embodiment of this application. See also... Figure 7 In one specific embodiment provided in this application, the export command for the first asset file is automatically generated by the application development engine. In response to entering the export command for the first asset file on the resource management interface, the first terminal 10 performs the following steps:

[0148] S71. Build a preset system environment; for example, build a Windows environment.

[0149] S72. Update the application development engine; for example, this step may involve downloading new engine code or upgrading the existing engine version.

[0150] S73. In response to inputting an export command for the first asset file on the resource management interface, determine a first media memory region for storing the first asset file;

[0151] S74. Export the first asset file from the first media memory region in the application development engine's memory;

[0152] S75. If an export error occurs during the export process of the first asset file by the application development engine, an export error message will be generated and automatically sent to other terminals so that relevant personnel can view the export error message and resolve the export problem on other terminals.

[0153] In step 1200, in response to the input of a first import command for the second asset file on the resource management interface, the second asset file is imported into memory. Specifically, this includes: in response to the import operation of the second asset file executed on the resource management interface, the first terminal 10 generates a first import command for the second asset file; and in response to the first import command for the second asset file, the first terminal 10 imports the second asset file into the memory of the application development engine. This method enables the automatic import function of asset files in the application development engine.

[0154] Figure 8 See the flowchart illustrating the process of importing a second asset file provided in this application embodiment. Figure 8 In one specific embodiment provided in this application, the first import instruction for the second asset file is automatically generated by the application development engine. In response to entering the first import instruction for the second asset file on the resource management interface, the first terminal 10 performs the following steps:

[0155] S81, Build a preset system environment; for example, build a Windows environment.

[0156] S82. Update the application development engine; for example, this step may involve downloading new engine code or upgrading the existing engine version.

[0157] S83. In response to the input of a first import command for the second asset file on the resource management interface, a second media memory region for storing the second asset file is determined.

[0158] S84. Import a second asset file into the second media memory region in the application development engine's memory. The second asset file is in a format that the application development engine can recognize.

[0159] S85. If the application development engine encounters a first import exception during the import process of the second asset file, a first export exception message will be generated and automatically sent to other terminals so that relevant personnel can view the first export exception message and resolve the first import problem on other terminals.

[0160] In step 1300, in response to the input of a second import command for the first audio resource on the resource management interface, the first audio resource is imported into memory and a third asset file corresponding to the first audio resource is generated. Specifically, in response to the import operation of the first audio resource performed by the business personnel on the resource management interface, the first terminal 10 generates a second import command for the first audio resource. In response to the second import command for the first audio resource, the first terminal 10 imports the first audio resource into the memory of the application development engine and generates a third asset file corresponding to the first audio resource.

[0161] Figure 9 The flowchart illustrating the import of the first audio resource provided in this application embodiment is shown below. Figure 9 In one specific embodiment provided in this application, the first audio resource is a voice resource. In response to inputting a second import command for the first audio resource on the resource management interface, the first terminal 10 performs the following steps:

[0162] S91. Build a preset system environment; for example, build a Windows environment.

[0163] S92. Update the application development engine; for example, this step may involve downloading new engine code or upgrading the existing engine version.

[0164] S93. In response to the input of a second import command for the first audio resource on the resource management interface, the first audio resource is imported into the memory of the application development engine.

[0165] S94, The application development engine performs preset processing on the first audio resource; for example, it performs editing, dubbing, and other processing on the first audio resource.

[0166] S95. Inspect the processed first audio resource; for example, perform a dialogue check, where Automatic Speech Recognition (ASR) can be used to inspect the processed first audio resource.

[0167] S96. Generate a third asset file corresponding to the first audio resource. The third asset file may include the processed first audio resource, the inspection results of the first audio resource, etc.

[0168] S97. If a second import error occurs during the import process of the first audio resource, the application development engine will generate a second export error message and automatically send the second export error message to other terminals so that relevant personnel can view the second export error message and resolve the second export problem on other terminals.

[0169] In some embodiments, the method further includes step 1110:

[0170] Step 1110: Create the main thread, the asynchronous loading thread, and the audio processing thread respectively. Establish a communication connection between the main thread and the asynchronous loading thread, and establish a communication connection between the main thread and the audio processing thread.

[0171] The main thread is used to respond to audio playback requests from target audio events, check if the target asset file for the target audio event exists in the audio event pool, and unload the target audio resource by calling the playback end callback function when the target audio event ends. The asynchronous loading thread is used to asynchronously load the target asset file for the target audio event and load the target audio resource referenced in the target asset file from the application development engine's memory. The audio processing thread is used to register the target audio event by calling the audio engine's application interface through the playback start callback function in order to play the target audio resource.

[0172] Specifically, when the first terminal 10 processes audio resources, it creates a main thread, an asynchronous loading thread, and an audio processing thread. The main thread establishes a communication connection with the asynchronous loading thread, and the main thread establishes a communication connection with the audio processing thread. By using multiple threads to process audio resources together, the efficiency of audio resource processing is improved.

[0173] Figure 10 This is a flowchart illustrating the multi-threaded playback method for target audio resources provided in an embodiment of this application. (See attached diagram.) Figure 10 ,include:

[0174] S101. The main thread responds to the audio playback request of the target audio event and checks whether the target asset file of the target audio event exists in the audio event pool; if not, proceed to step S102; if yes, proceed to step S105.

[0175] S102. If the target asset file for the target audio event does not exist in the audio event pool, the main thread sends a request to the asynchronous loading thread to obtain the target asset file; if the target asset file for the target audio event exists in the audio event pool, the main thread sends an audio playback request to the audio processing thread.

[0176] S103. The asynchronous loading thread responds to the target asset file retrieval request, loads the target asset file of the target audio event, and loads the target audio resources referenced in the target asset file from the memory of the application development engine.

[0177] S104. The main thread retrieves the loading result from the asynchronous loading thread; if the main thread successfully retrieves the loading result from the asynchronous loading thread, proceed to step S105; if the main thread fails to retrieve the loading result from the asynchronous loading thread, proceed to step S108.

[0178] S105. The main thread sends an audio playback request to the audio processing thread; for example, if the target asset file of the target audio event exists in the audio event pool, the main thread sends an audio playback request to the audio processing thread; or, if the main thread successfully obtains the loading result from the asynchronous loading thread, the main thread sends an audio playback request to the audio processing thread.

[0179] S106. The audio processing thread responds to the audio playback request by calling the audio engine's application interface through the playback start callback function to register the target audio event and play the target audio resource.

[0180] S107. After playback ends, the audio processing thread sends a playback end message to the main thread indicating that playback has ended.

[0181] S108. The main thread calls the playback end callback function to end the playback process;

[0182] S109. The main thread calls the playback end callback function to unload the target audio resource.

[0183] In some embodiments, the playback interface for playing the target audio resource is one of the preset playback interfaces, and the preset playback interface includes at least the following first playback interface, second playback interface, third playback interface, and fourth playback interface:

[0184] The first playback interface is used to play two-dimensional sound effects at the origin position in virtual three-dimensional space.

[0185] The first playback interface is suitable for 2D playback scenarios. Audio resources played through the first playback interface do not change sound effects with spatial position during playback, as if they were playing at the origin (or default position). In game scenarios, ambient sounds, background music, or sound effects that do not require spatial positioning effects can be played through the first playback interface. For example, the function of the first playback interface can be represented as static AcSoundIDPlaySound(const FString&AkEventName,const FAcSoundCallback*SoundCallback=nullptr,bool ForceSyncLoad=false).

[0186] The second playback interface is used to play 3D sound effects in the virtual 3D space, following the overall position changes of the virtual character.

[0187] The second playback interface allows audio resources to be associated with virtual actors (Actors), and these effects move in space as the virtual actors move, making it suitable for 3D scenes. The second playback interface provides a more realistic sound experience because the source location of the sound effects changes with the virtual actor's position. For example, the function of the second playback interface can be represented as `static AcSoundIDPlaySound(const FString&AkEventName,AActor*Actor,const FAcSoundCallback*SoundCallback=nullptr,bool ForceSyncLoad=false)`.

[0188] The third playback interface is used to play 3D sound effects in the virtual 3D space, following the local positional changes of the target part of the virtual character.

[0189] The third playback interface allows the playback of audio resources to be associated with the local position of a target part of a virtual character. For example, if the virtual character is a human, the target part can be a specific body part, such as the head, hand, leg, or foot. The third playback interface provides fine-grained control over the playback position of sound effects, allowing the sound effects to move with a specific part of the virtual character. For instance, footsteps can be played through the third playback interface, and the footstep sound effect can move with the foot bones as the foot bones move. For example, the function of the third playback interface can be represented as StaticAcSoundIDPlaySound(constFString&AkEventName,UAkComponent*AkComponent,constFAcSoundCallback*SoundCallback=nullptr,boolForceSyncLoad=false).

[0190] The fourth playback interface is used to play 3D sound effects at specified coordinates in virtual 3D space.

[0191] The fourth playback interface controls sound effect playback by precisely setting specified coordinates when playing audio resources. This can be achieved by setting the playback position (x, y, z coordinates) of the sound effect through API calls. For example, the function of the fourth playback interface can be represented as `static AcSoundIDPlaySound(const FString&AkEventName, FVectorLocation, FRotator Rotator, const FAcSoundCallback *SoundCallback = nullptr, boolForceSyncLoad = false)`.

[0192] This application designs multiple preset playback interfaces to provide different audio playback modes for audio resources, thereby meeting the different needs of various scenarios in application development.

[0193] In some embodiments, the playback start callback function and playback end callback function are not triggered within the preset playback interface.

[0194] Specifically, when an audio resource is in playback mode, playback start and playback end callbacks are not triggered. The triggering of callback functions (such as playback start and playback end callbacks) is asynchronous; that is, they are triggered after the preset playback interface returns a function value indicating the playback result, based on the actual progress of the audio resource playback (such as when the audio resource actually starts playing or when playback ends). This design allows code calling the preset playback interface to safely associate and identify subsequent callback functions based on the preset playback interface's return value, without receiving callbacks during the execution of the preset playback interface.

[0195] In some embodiments, the playback start callback function is triggered before the audio engine's default callback function is triggered. This approach allows users to perform necessary initialization or setup tasks that depend on the audio resource having been successfully loaded, before the audio resource actually begins playing.

[0196] In some embodiments, after the preset playback interface returns a valid function value, a playback end callback function is triggered. That is, if the preset playback interface successfully returns a valid function value (indicating that the sound playback request has been accepted), then regardless of the final result of the sound playback (including normal playback end, premature stopping, etc.), a playback end callback function should be triggered. This allows relevant personnel to receive information that the audio resource playback has ended, thereby enabling them to perform cleanup tasks, update the game state, or trigger other logic.

[0197] In some embodiments, if a pause playback function is called when the target audio resource is loaded, a playback end callback function is triggered.

[0198] In other words, if the playback stop function is called to stop playback while the target audio resource is loading but has not yet started playing, the playback end callback function will be triggered to send a relevant notification to inform the audio engine of the relevant information.

[0199] In some embodiments, the method further includes steps 1120-1130:

[0200] Step 1120: In response to an audio pause request from the target audio resource, if the target audio resource is in playback mode, pause playback of the target audio resource by calling the audio engine's application programming interface; or

[0201] Step 1130: In response to the audio pause request of the target audio resource, if the target audio resource is in a loading state, continue loading the target audio resource until loading is complete, and pause the playback of the target audio resource by calling the application interface of the audio engine according to the audio pause request.

[0202] In response to an audio pause request from the target audio resource, if the target audio resource is currently playing, the application development engine calls the audio engine's application programming interface to cause the audio engine to pause playback of the target audio resource.

[0203] In response to an audio pause request from the target audio resource, if the target audio resource is in a loading state at this time, the application development engine continues to load the target audio resource until loading is complete, and calls the audio engine's application programming interface to immediately stop the playback of the target audio resource.

[0204] In some embodiments, the method further includes steps 1140-1150:

[0205] 1140. Display the debugging management interface corresponding to the target application through the application development engine;

[0206] 1150. In response to triggering the audio debugging switch to be turned on on the debugging management interface, display the audio debugging window of the target application.

[0207] The target application can be any application that requires animation, such as a game application or an animation production application.

[0208] Please see Figure 11 and Figure 12 , Figure 11 This is an example diagram of the management and debugging interface provided in the embodiments of this application. Figure 12 This is an example diagram of an audio debugging window provided in an embodiment of this application. The debugging management interface of the target application is used to debug audio events used in the target application. The debugging management interface includes multiple audio events, such as... Figure 11The system includes animal sounds, skill sounds, breathing sounds, and 2D sounds (2DSound). Each audio event corresponds to an audio debugging switch. In response to a user's activation of the audio debugging switch for a specific audio event on the debugging management interface, the first terminal 10 will display the audio debugging window for that audio event (e.g., ...). Figure 12 As shown in the image, relevant personnel can perform audio debugging operations on this audio event used in the target application in this audio debugging window.

[0209] In some embodiments, the method further includes steps 1160-1170:

[0210] Step 1160: Display the animation development interface corresponding to the target application through the application development engine. The animation development interface is configured with an animation notification status interface, which is used to trigger the corresponding audio event in the target state of the animation in the target application; or include the following steps 1170.

[0211] The animation development interface is a visual interface for relevant personnel to operate when developing animations.

[0212] The animation notification status interface includes multiple configurable parameters. These parameters can be set to different values ​​for different animations to meet the different design requirements of different animations.

[0213] In one specific embodiment provided in this application, the parameters of the animation notification status interface may include audio event name, audio event, attachment point (playback position of audio resource), stop audio upon termination / abortion (whether the playback of the associated audio resource should be stopped when the animation / skill is terminated or aborted), trigger only when the object is the main character (in some cases, the main character and other players will use the same audio event, but if other players do not want the audio resource corresponding to the audio event to be played, this attribute can be used), terrain material related (different material types or terrain types correspond to different sound effects), automatic 3P addition (the timbre of a player who is not the main character when using this audio event is different from the timbre of the main character player), use timbre (different types of characters correspond to different timbre), etc.

[0214] The target state of the animation can be a specific state in which the target character in the animation is in a running state, a jumping state, or an attack, or it can be a specific time period or a key frame in the animation, such as a period of time after the animation starts or before it ends.

[0215] Figure 13 A schematic diagram of the animation development interface provided in this application embodiment is shown below. Figure 13 , Figure 13 The animation development interface shown has multiple animation notification status interfaces configured, including: the event name of the audio event (Play_100203010), the audio event (i.e., the asset file corresponding to the audio event name), the attachment point, stopping the audio when terminating / aborting, triggering only when the owner (object) is the main character, terrain material related, automatically adding 3P, using timbre and other parameters. For a certain parameter, the parameter can be set by checking the check box after the parameter.

[0216] Step 1170: Display the skill editor corresponding to the target application through the application development engine. The skill editor is configured with a skill playback audio task, which is used to play the audio resource corresponding to the target skill when the target skill is executed.

[0217] The skill editor is used to design skills for a target application, such as skill sound effects and special effects.

[0218] The skill playback audio task also has several configurable parameters. By setting the parameters of the skill playback audio task, you can design the corresponding audio effect when the target skill is released, so that the target skill achieves the expected effect.

[0219] See Figure 14 , Figure 14 This is an example diagram of the interface of the skill editor provided in an embodiment of this application. The skill editor allows setting parameters for the skill playback audio task, thereby designing the sound effects when the target skill is cast. For example... Figure 14 As shown, the configurable parameters for the skill playback audio task include: attachment point, audio, automatic stop upon task completion, trigger only when the owner is the main character, terrain and material related [downward ray detection (emitting a ray downwards from the bottom of the character to detect terrain or materials)], automatically adding '_3P' as the event name to trigger, using timbre, playing near the warning location (playing the corresponding audio resource when the target skill is released at the warning location), playing at the skill target location, and whether it only takes effect under specific buffs. For a specific parameter, you can set the parameter by checking the checkbox after the parameter.

[0220] In some embodiments, the method further includes steps 1180-1190:

[0221] Step 1180: Establish the path mapping relationship between the event names of each audio event and the simplified path;

[0222] Step 1190: Before playing the target audio resource, load the target audio resource through the path mapping relationship.

[0223] Because the path to the audio resource of an audio event is relatively long, loading the audio resource will take a long time and occupy a large amount of storage space. Therefore, the path of the audio resource of the audio event can be simplified. This application does not limit the method of path simplification, as long as the simplified path of the audio resource of the audio event uniquely corresponds to the audio event, and the audio resource of the audio event can be found based on the simplified path.

[0224] After establishing the path mapping relationship between the time name of the audio event and the simplified path, when loading the target audio resource, the simplified path corresponding to the target mapped resource can be obtained simply by looking up the path mapping relationship table, and then the location of the target audio resource can be retrieved based on the simplified path.

[0225] In a specific embodiment provided in this application, the event name of an audio event and the list of paths where the audio event is located can be used as the simplified path of the audio event (in this case, the event names of audio events cannot be duplicated under a path list). For example, the path of an audio event before simplification is: / GameEvents / AMB / 2D / Play_Amb_2D_Crop.Play_Amb_2D_Crop, where 2D is the event name of the audio event and AMB is the list of paths where the audio resources of the audio event are located. Then, the simplified path of the audio event 2D can be AMB / 2D. When loading the target audio resource of the target audio event 2D, the path list where the target audio resource of the target audio event 2D is located can be determined as AMB according to the path mapping relationship. Further, the audio resource with the index 2D can be found in the list AMB.

[0226] All of the above technical solutions can be combined in any way to form optional embodiments of this application, and will not be described in detail here.

[0227] This embodiment of the application, in response to an audio playback request for a target audio event, checks if a target asset file for the target audio event exists in the audio event pool. If the target asset file for the target audio event does not exist in the audio event pool, it asynchronously loads the target asset file and loads the target audio resources referenced in the target asset file from the application development engine's memory. The memory is used to store asset files for multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources, and the asset files are generated based on an event packaging mechanism. The target audio event is registered by calling the audio engine's application programming interface to play the target audio resources. When the target audio event ends, the target audio resources are unloaded. This embodiment of the application generates asset files containing audio resources based on an event packaging mechanism and synchronizes them to the application development engine. It also introduces the concept of an audio event pool, realizing on-demand loading and unloading of audio resources in the audio playback process, effectively reducing audio memory consumption and significantly improving the management and utilization efficiency of audio resources.

[0228] To facilitate better implementation of the audio resource processing method of this application, this application also provides an audio resource processing apparatus. Please refer to... Figure 15 , Figure 15 This is a schematic diagram of the structure of an audio resource processing apparatus provided in an embodiment of this application. The audio resource processing apparatus 200 may include:

[0229] The first processing unit 210 is used to, in response to an audio playback request of a target audio event, search the audio event pool to see if the target asset file of the target audio event exists.

[0230] The second processing unit 220 is used to asynchronously load the target asset file of the target audio event if the target asset file of the target audio event does not exist in the audio event pool, and load the target audio resources referenced in the target asset file from the memory of the application development engine. The memory is used to store asset files of multiple audio events synchronized from the audio engine to the application development engine. Each asset file includes at least audio resources. The asset file is generated based on the event packaging mechanism.

[0231] The third processing unit 230 is used to register target audio events by calling the application programming interface of the audio engine in order to play the target audio resources;

[0232] The unloading unit 240 is used to unload the target audio resource when the target audio event ends.

[0233] In some embodiments, the device is further configured to:

[0234] Displays the resource management interface of the application development engine;

[0235] In response to the asset synchronization command entered on the resource management interface, the asset files of multiple audio events in the audio engine are synchronized to the memory of the application development engine.

[0236] In some embodiments, before synchronizing asset files of multiple audio events in the audio engine to the memory of the application development engine, the apparatus is further configured to:

[0237] When a media memory region for storing asset files is detected in memory and is not empty, the media memory region is released according to the content release function corresponding to the configuration option.

[0238] When releasing media memory areas, clear the metadata in the data blocks;

[0239] Update memory usage statistics.

[0240] In some embodiments, the device is further configured to:

[0241] In response to an export command for the first asset file being entered on the resource management interface, the first asset file is exported from memory; or

[0242] In response to entering a first import command for the second asset file on the resource management interface, the second asset file is imported into memory; or

[0243] In response to the input of a second import command for the first audio resource on the resource management interface, the first audio resource is imported into memory, and a third asset file corresponding to the first audio resource is generated.

[0244] In some embodiments, the step of generating asset files based on an event packaging mechanism includes:

[0245] The audio engine divides audio resource data into multiple audio events, and each audio event includes at least one audio resource segment and the corresponding metadata of the audio resource segment;

[0246] The audio engine generates asset files for each audio event based on an event packaging mechanism. The asset files are in a format that the application development engine can recognize.

[0247] In some embodiments, the device is further configured to:

[0248] Create a main thread, an asynchronous loading thread, and an audio processing thread respectively. Establish a communication connection between the main thread and the asynchronous loading thread, and establish a communication connection between the main thread and the audio processing thread.

[0249] The main thread is used to respond to audio playback requests for the target audio event, check if the target asset file for the target audio event exists in the audio event pool, and unload the target audio resource by calling the playback end callback function when the target audio event ends.

[0250] The asynchronous loading thread is used to asynchronously load the target asset file of the target audio event and load the target audio resources referenced in the target asset file from the memory of the application development engine;

[0251] The audio processing thread is used to register target audio events by calling the audio engine's application interface through the playback start callback function, in order to play the target audio resource.

[0252] In some embodiments, the playback interface for playing the target audio resource is one of the preset playback interfaces, which includes:

[0253] The first playback interface is used to play two-dimensional sound effects at the origin position in virtual three-dimensional space;

[0254] The second playback interface is used to play three-dimensional sound effects in the virtual three-dimensional space, following the overall position changes of the virtual character.

[0255] The third playback interface is used to play three-dimensional sound effects in the virtual three-dimensional space, following the local positional changes of the target part of the virtual character.

[0256] The fourth playback interface is used to play 3D sound effects at specified coordinates in virtual 3D space.

[0257] In some embodiments, the device is further configured to:

[0258] In response to an audio pause request from the target audio resource, if the target audio resource is in playback mode, pause playback of the target audio resource by calling the audio engine's application programming interface; or

[0259] In response to an audio pause request from the target audio resource, if the target audio resource is in a loading state, continue loading the target audio resource until loading is complete, and pause playback of the target audio resource by calling the application interface of the audio engine according to the audio pause request.

[0260] In some embodiments, the device is further configured to:

[0261] The application development engine displays the debugging and management interface corresponding to the target application.

[0262] In response to triggering the audio debugging switch to be turned on in the debugging management interface, the audio debugging window of the target application is displayed.

[0263] In some embodiments, the device is further configured to:

[0264] The application development engine displays the animation development interface corresponding to the target application. This interface is configured with an animation notification status interface, which is used to trigger corresponding audio events when the animation reaches a specific state within the target application.

[0265] The application development engine displays the skill editor corresponding to the target application. The skill editor is configured with a skill playback audio task, which is used to play the audio resource corresponding to the target skill when the target skill is executed.

[0266] In some embodiments, the device is further configured to:

[0267] Establish a path mapping relationship between the event names of each audio event and the simplified path;

[0268] Before playing the target audio resource, load the target audio resource through the path mapping relationship.

[0269] It should be noted that the functions of each module in the audio resource processing device 200 in this application embodiment can be referred to the specific implementation of any embodiment in the above method embodiments, and will not be repeated here.

[0270] Each unit in the aforementioned device can be implemented entirely or partially through software, hardware, or a combination thereof. Each unit can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can invoke and execute the corresponding activation operation of each unit.

[0271] For example, the audio resource processing device 200 can be integrated into a terminal or server that has storage and a processor and thus computing power, or the audio resource processing device 200 can be the terminal or server.

[0272] In some embodiments, this application also provides a computer device including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above-described method embodiments.

[0273] Figure 16 A schematic structural diagram of the computer device provided in the embodiments of this application, such as Figure 16As shown, the computer device 300 may include: a communication interface 301, a memory 302, a processor 303, and a communication bus 304. The communication interface 301, memory 302, and processor 303 communicate with each other via the communication bus 304. The communication interface 301 is used for data communication between the device 300 and external devices. The memory 302 can be used to store software programs and modules, and the processor 303 runs the software programs and modules stored in the memory 302, such as the software programs for the corresponding operations in the aforementioned method embodiments.

[0274] In some embodiments, the processor 303 may invoke software programs and modules stored in the memory 302 to perform the following operations: in response to an audio playback request for a target audio event, check if a target asset file for the target audio event exists in the audio event pool; if no target asset file for the target audio event exists in the audio event pool, asynchronously load the target asset file for the target audio event, and load the target audio resources referenced in the target asset file from the memory of the application development engine, wherein the memory is used to store asset files for multiple audio events synchronized from the audio engine to the application development engine, each asset file including at least audio resources, and the asset files are generated based on an event packaging mechanism; register the target audio event by calling the application programming interface of the audio engine to play the target audio resources; and unload the target audio resources when the target audio event ends.

[0275] In some embodiments, the computer device 300 may be integrated into a terminal or server that has storage and a processor, thus possessing computing capabilities; or the computer device 300 may be the terminal or server. The terminal may be a smartphone, tablet, laptop, smart TV, smart speaker, wearable smart device, personal computer, or other similar device. The server may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms.

[0276] This application also provides a computer-readable storage medium for storing a computer program. This computer-readable storage medium can be applied to a computer device, and the computer program causes the computer device to execute the corresponding processes in the methods described above in the embodiments of this application; for brevity, further details are omitted here.

[0277] This application also provides a computer program product including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0278] This application also provides a computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the corresponding processes in the methods described above in the embodiments of this application. For brevity, these details will not be elaborated further here.

[0279] It should be understood that the processor in the embodiments of this application may be an integrated circuit chip with signal processing capabilities. In implementation, the steps of the above method embodiments can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor described above can be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method.

[0280] It is understood that the memory in the embodiments of this application can be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchlink DRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory used in the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0281] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0282] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0283] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0284] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0285] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0286] In addition, the functional units in the embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0287] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer or a server) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0288] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio resource processing method, characterized in that, The method comprises: in response to an audio playback request of a target audio event, searching for a target asset file of the target audio event in an audio event pool; if the target asset file of the target audio event does not exist in the audio event pool, asynchronously loading the target asset file of the target audio event, and loading a target audio resource referenced in the target asset file from a memory of an application development engine, the memory being used to store asset files of a plurality of audio events synchronized from an audio engine to the application development engine, each of the asset files comprising at least an audio resource, the asset files being generated based on an event packaging mechanism; registering the target audio event by calling an application program interface of the audio engine to play the target audio resource; when the target audio event ends, unloading the target audio resource.

2. The audio resource processing method of claim 1, wherein, The method further comprises: displaying a resource management interface of the application development engine; in response to an asset synchronization instruction input on the resource management interface, synchronizing asset files of a plurality of audio events in the audio engine to the memory of the application development engine.

3. The audio resource processing method of claim 2, wherein, Before synchronizing the asset files of the plurality of audio events in the audio engine to the memory of the application development engine, the method further comprises: when detecting that a media memory area in the memory has been allocated for storing the asset files and the media memory area is not empty, selecting a corresponding content release function according to a configuration option to release the media memory area; when releasing the media memory area, clearing metadata in a data block; updating usage statistical information of the memory.

4. The audio resource processing method of claim 2, wherein, The method further comprises: in response to an export instruction for a first asset file input on the resource management interface, exporting the first asset file from the memory; or in response to a first import instruction for a second asset file input on the resource management interface, importing the second asset file to the memory; or in response to a second import instruction for a first audio resource input on the resource management interface, importing the first audio resource to the memory and generating a third asset file corresponding to the first audio resource.

5. The audio resource processing method of claim 2, wherein, The step of generating the asset files based on the event packaging mechanism comprises: the audio engine divides audio resource data into a plurality of audio events, each of the audio events comprising at least one audio resource segment and metadata corresponding to the audio resource segment; the audio engine generates an asset file of each of the audio events based on the event packaging mechanism, the asset file being in a format recognizable by the application development engine.

6. The audio resource processing method of claim 1, wherein, The method further comprises: respectively creating a main thread, an asynchronous loading thread and an audio processing thread, establishing a communication connection between the main thread and the asynchronous loading thread, and establishing a communication connection between the main thread and the audio processing thread; the main thread is configured to search for a target asset file of a target audio event in an audio event pool in response to an audio playback request of the target audio event, and to unload a target audio resource by calling a playback end callback function when the target audio event ends; The asynchronous loading thread is configured to asynchronously load a target asset file of the target audio event and load a target audio resource referenced in the target asset file from a memory of an application development engine; The audio processing thread is configured to register the target audio event by calling an application program interface of the audio engine through a play start callback function, so as to play the target audio resource.

7. The audio resource processing method of claim 6, wherein, The play interface for playing the target audio resource is one of preset play interfaces, and the preset play interfaces include: A first play interface configured to play a two-dimensional sound effect at an origin position in a virtual three-dimensional space; A second play interface configured to play a three-dimensional sound effect following a global position change of a virtual character in the virtual three-dimensional space; A third play interface configured to play a three-dimensional sound effect following a local position change of a target part of the virtual character in the virtual three-dimensional space; A fourth play interface configured to play a three-dimensional sound effect at a specified coordinate position in the virtual three-dimensional space.

8. The audio resource processing method of claim 1, wherein, The method further includes: In response to an audio pause request of the target audio resource, if the target audio resource is in a playing state, pausing playing of the target audio resource by calling an application program interface of the audio engine; or In response to the audio pause request of the target audio resource, if the target audio resource is in a loading state, continuing to load the target audio resource until the loading is completed, and pausing playing of the target audio resource according to the audio pause request by calling the application program interface of the audio engine.

9. The audio resource processing method of claim 1, wherein, The method further includes: Displaying a debugging management interface corresponding to the target application by the application development engine; In response to an opening operation of an audio debugging switch triggered on the debugging management interface, displaying an audio debugging window of the target application.

10. The audio resource processing method of claim 9, wherein, The method further includes: Displaying an animation development interface corresponding to the target application by the application development engine, the animation development interface being configured with an animation notification state interface, the animation notification state interface being configured to trigger a corresponding audio event in a target state of an animation in the target application; or Displaying a skill editor corresponding to the target application by the application development engine, the skill editor being configured with a skill play audio task, the skill play audio task being configured to play an audio resource corresponding to a target skill when the target skill is executed.

11. The audio resource processing method of claim 1, wherein, The method further includes: Establishing a path mapping relationship between an event name of each audio event and a simplified path; Loading the target audio resource through the path mapping relationship before playing the target audio resource.

12. An audio resource processing apparatus, characterized by comprising: The apparatus includes: A first processing unit configured to, in response to an audio play request of a target audio event, find whether a target asset file of the target audio event exists in an audio event pool; a second processing unit, configured to load a target asset file of the target audio event if the target asset file of the target audio event does not exist in the audio event pool, and load a target audio resource referenced in the target asset file from a memory of an application development engine, the memory being configured to store asset files of a plurality of audio events synchronized from an audio engine to the application development engine, each of the asset files comprising at least an audio resource, the asset files being generated based on an event packaging mechanism; a third processing unit, configured to register the target audio event by calling an application program interface of the audio engine to play the target audio resource; an uninstalling unit, configured to uninstall the target audio resource when the target audio event ends.

13. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program is adapted to be loaded by a processor to execute the audio resource processing method in any one of claims 1-11.

14. A computer device, comprising: The computer device comprises a processor and a memory, and the memory stores a computer program, and the processor is configured to execute the audio resource processing method in any one of claims 1-11 by calling the computer program stored in the memory.

15. A computer program product comprising computer instructions, characterized in that, The computer instructions are executed by the processor to implement the audio resource processing method in any one of claims 1-11.

Images