Audio processing method, apparatus, medium, and electronic device

Abstract

The present disclosure relates to an audio processing method, device, medium and electronic equipment, and relates to the technical field of computers. The method obtains target audio generated by a target sound source in a virtual scene, determines a target loudness corresponding to the target audio according to a target distance between the target sound source and a virtual object in combination with a loudness decay curve, and outputs the target audio according to the target loudness. The loudness of the generated audio can be presented as a sound source with no fixed position effect in the virtual scene. The target audio generated by the sound source in the virtual scene can present a real hearing sensation, thereby creating a more real atmosphere in the virtual scene and enabling players to have a more immersive game experience.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, and more specifically, to an audio processing method, apparatus, medium, and electronic device. Background Technology

[0002] Ambient audio can be used to create a realistic gaming experience or enhance the game's atmosphere. For example, playing natural ambient audio such as birdsong and rustling grass in a virtual scene can make players feel as if they are in a real natural environment. Therefore, how to process and play ambient audio in virtual scenes to create a realistic gaming experience for players has become an increasingly important research direction. Summary of the Invention

[0003] This summary section is provided to briefly introduce the concepts, which will be described in detail in the detailed description section below. This summary section is not intended to identify key or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0004] In a first aspect, this disclosure provides an audio processing method, including:

[0005] Acquire the target audio generated by the target sound source in the virtual scene;

[0006] Based on the target distance between the target sound source and the virtual object, and combined with the loudness decay curve, the target loudness corresponding to the target audio is determined. The loudness decay curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, and in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance.

[0007] The target audio is output based on the target loudness.

[0008] Secondly, this disclosure provides an audio processing apparatus, comprising:

[0009] The acquisition module is configured to acquire target audio generated by a target sound source in a virtual scene.

[0010] The determination module is configured to determine the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, combined with a loudness attenuation curve. The loudness attenuation curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, and in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance.

[0011] The output module is configured to output the target audio based on the target loudness.

[0012] Thirdly, this disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of the method described in the first aspect.

[0013] Fourthly, this disclosure provides an electronic device, comprising:

[0014] A storage device on which computer programs are stored;

[0015] A processing device for executing the computer program in the storage device to implement the steps of the method described in the first aspect.

[0016] Based on the above technical solution, by acquiring the target audio generated by the target sound source in the virtual scene, determining the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, and combining the loudness attenuation curve, and outputting the target audio based on the target loudness, the loudness of the generated audio can be presented to the user's auditory system as a sound source without a fixed position effect. The target audio generated in the virtual scene can present a realistic listening experience, thereby creating a more realistic atmosphere in the virtual scene and enabling players to have a more immersive gaming experience.

[0017] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0018] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and the originals and elements are not necessarily drawn to scale. In the drawings:

[0019] Figure 1 This is a flowchart illustrating an audio processing method according to some embodiments.

[0020] Figure 2 This is a schematic diagram of a loudness decay curve shown according to some embodiments.

[0021] Figure 3 This is a schematic diagram of the structure of an audio processing device according to some embodiments.

[0022] Figure 4 This is a schematic diagram of the structure of an electronic device according to some embodiments. Detailed Implementation

[0023] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0024] It should be understood that the steps described in the method embodiments of this disclosure may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this disclosure is not limited in this respect.

[0025] The term "comprising" and its variations as used herein are open-ended inclusions, meaning "including but not limited to". The term "based on" means "at least partially based on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Definitions of other terms will be given in the description below.

[0026] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0027] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0028] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0029] Figure 1 This is a flowchart illustrating an audio processing method according to some embodiments. For example... Figure 1As shown, this disclosure provides an audio processing method that can be executed by an electronic device, specifically by an audio processing device. This device can be implemented in software and / or hardware and configured within the electronic device. Figure 1 As shown, the method may include the following steps.

[0030] In step 110, the target audio generated by the target sound source in the virtual scene is obtained.

[0031] Here, virtual scene can refer to virtual reality scene, such as virtual reality scene presented using VR device. Of course, the virtual scene can also be other video game scene besides virtual reality scene.

[0032] Electronic devices can acquire the target audio generated by a target sound source in response to a command to play the target audio from that sound source in a virtual scene. For example, when the target audio of wind sound needs to be played in a virtual scene, the target audio of wind sound is acquired. It should be understood that target audio refers to the unprocessed raw audio signal emitted in the real world by the target sound source, which is pre-made by game developers.

[0033] The target sound source is a sound source whose loudness, as perceived by the user's auditory system, has no fixed location. That is, the loudness of the audio produced by the target sound source is not perceived from a fixed location within the virtual scene, but rather varies in loudness depending on the distance between the player character and the target sound source. For example, the target audio could be the sound of wind. A valley in the virtual scene might have wind; when the player character is far from the valley, the loudness of the wind is greater, while when the player character is closer, the loudness of the wind decreases or even becomes zero. Similarly, the target audio could be thunder. For instance, when a player hears thunder coming from a cloud in the distance, they cannot pinpoint which cloud is producing the thunder when they move closer to it.

[0034] It should be understood that, generally, the loudness of a sound source placed in a fixed location within a virtual scene is constant. As the player character approaches the sound source, the loudness of the sound heard by the player increases; conversely, as the player character moves away from the sound source, the loudness of the sound heard by the player decreases. For example, with a waterfall in a virtual scene, the loudness of the waterfall's sound heard by the player increases as the player approaches it, and decreases as the player character moves away from it. In other words, the loudness of the waterfall's sound heard by the player has an effect on the user's auditory system, corresponding to its fixed location within the virtual scene.

[0035] However, the target sound source in this embodiment differs from a sound source like a waterfall, which is located in a fixed position. The loudness of the target audio generated by the target sound source is only sufficient for the player's auditory system to discern the approximate location and direction of the target audio within the virtual scene, but not its specific location. For example, regarding the sound of wind in a valley, if the wind is blowing in the valley, the player will hear a louder sound when the character is far from the valley. However, as the player approaches the valley, they can feel the wind, but the loudness of the sound will decrease.

[0036] It's worth noting that although the target sound source (equivalent to the sound emitter) corresponding to the target audio (wind sound) is placed in a fixed location within the virtual scene (such as a fixed location in a valley), the loudness of the target audio in the player's auditory system is not emitted from that fixed location. In other words, the audio effect, which appears to have no fixed location, is presented in the player's auditory system through variations in loudness.

[0037] It's important to clarify that a sound source refers to an object in a virtual scene that emits a specific sound effect. For example, if a waterfall in a virtual scene emits a waterfall sound, then there will be a sound source at the location of the waterfall, and that sound source will emit the waterfall sound. It should be understood that a sound source can be invisible in a virtual scene; in other words, a sound source simply represents the sound event of an object playing a sound effect, and it is not necessarily a physical object. Additionally, in other terminology, a sound source can also be referred to as an emitter.

[0038] In step 120, the target loudness corresponding to the target audio is determined based on the target distance between the target sound source and the virtual object, combined with the loudness attenuation curve.

[0039] Here, the virtual object can refer to a player character controlled by the player. In the loudness decay curve, distance and loudness correspond one-to-one, that is, different distances correspond to a loudness value.

[0040] In this disclosure, the target distance between the target sound source and the virtual object can be determined based on the location information of the target sound source and the virtual object in the virtual scene. Then, based on the target distance, the target loudness corresponding to the target distance is found in the loudness attenuation curve.

[0041] The loudness decay curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, while in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance.

[0042] It is worth noting that the starting point of the second distance interval can coincide with the ending point of the first distance interval, and the loudness corresponding to the starting point of the second distance interval can be the same as the loudness corresponding to the ending point of the first distance interval.

[0043] Figure 2 This is a schematic diagram of loudness decay curves shown according to some embodiments. For example... Figure 2 As shown, the horizontal axis of the loudness decay curve represents the distance between the virtual object and the target sound source, and the vertical axis represents the distance between them. In this embodiment, the unit of distance can be determined based on different game engines. For example, the unit of distance can be meters or centimeters. It should be noted that in game scenes, the unit of distance is actually a scale, which differs somewhat from the unit of distance in the real world. Additionally, the unit of loudness can be dB (decibels).

[0044] It should be understood that a loudness of 0 dB represents the maximum loudness, and a loudness of -200 dB represents the minimum loudness. In the first distance interval [0, 5000], the loudness of the audio produced by the target sound source is positively correlated with distance; that is, as the distance increases, the loudness of the target audio gradually increases from the minimum loudness to the maximum loudness. In the second distance interval (5000, 8000], the loudness of the audio produced by the target sound source is negatively correlated with distance; that is, as the distance increases, the loudness of the target audio gradually decreases from the maximum loudness to the minimum loudness.

[0045] It is worth noting that, Figure 2 The loudness values ​​shown are merely examples. In practical applications, other loudness values ​​can be used, such as the range of -192dB to +10dB, or the range of -80dB to 0dB. Here, -192dB and -80dB represent the minimum loudness, and +10dB and 0dB represent the maximum loudness. Of course, Figure 2 The relationship between distance and loudness value shown can also be adjusted according to the actual application.

[0046] It's important to note that loudness decay curves are actually used to simulate the sound effect of audio from a target sound source in a virtual scene in the real world. For example, if the target audio is wind, the loudness decay curve is used to simulate the effect of wind in the real world within the virtual scene.

[0047] In step 130, the target audio is output based on the target loudness.

[0048] Here, electronic devices can play target audio according to the target loudness. It should be understood that when the distance between the virtual object and the target sound source is different, the loudness of the target audio produced by the target sound source will be different for the player's ears.

[0049] pass Figure 2 The loudness decay curves shown indicate that when the distance between the virtual object controlled by the player and the target sound source generating the wind is within the first distance interval, the loudness of the wind sound perceived by the player's auditory system gradually increases from zero to a maximum loudness as the distance increases. When the distance between the virtual object controlled by the player and the target sound source generating the wind sound is within the second distance interval, the loudness of the wind sound perceived by the player's auditory system gradually decreases from a maximum loudness until it becomes zero as the distance increases. Therefore, using the above loudness decay curves, the auditory experience of a target sound source like wind sound in a virtual scene can be simulated as it would be in the real world.

[0050] Therefore, by acquiring the target audio generated by the target sound source in the virtual scene, determining the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, and combining the loudness attenuation curve, and outputting the target audio based on the target loudness, the loudness of the generated audio can be presented to the user's auditory system as a sound source without a fixed position effect. The target audio generated in the virtual scene can present a realistic listening experience, thereby creating a more realistic atmosphere in the virtual scene and enabling players to have a more immersive gaming experience.

[0051] In some feasible implementations, the first distance interval includes a first sub-interval and a second sub-interval, which increase sequentially. In the first sub-interval, the loudness corresponding to different distances is consistent. In the second sub-interval, the loudness of the audio generated by the target sound source is positively correlated with the distance.

[0052] Here, the end point of the first sub-interval can coincide with the starting point of the second sub-interval, and the loudness corresponding to the starting point of the second sub-interval can be the same as the loudness corresponding to the end point of the first sub-interval.

[0053] like Figure 2 As shown, the first distance interval [0, 5000] can be divided into a first sub-interval [0, 1800] and a second sub-interval (1800, 5000]. Within the first sub-interval, the loudness is consistent across different distances. That is, within the first sub-interval, regardless of the distance between the virtual object and the target sound source, the loudness remains constant, for example, remaining at -200dB, i.e., the minimum loudness. Within the first sub-interval, it indicates that the virtual object is close to the target sound source, and the target audio produced by the target sound source is presented to the player's auditory system with a low loudness or even inaudibility. As the virtual object gradually moves away from the target sound source and falls within the second sub-interval, the loudness of the target audio gradually increases and reaches its maximum. In other words, as the virtual object controlled by the player moves further away from the target sound source, the loudness of the target audio heard by the player gradually increases and eventually reaches its maximum.

[0054] Therefore, by dividing the virtual object into a first sub-interval and a second sub-interval, when the virtual object is located within a certain range (first sub-interval) of the target sound source, the loudness of the target audio heard by the player is relatively small or even inaudible, thus simulating a more realistic listening experience of the target audio.

[0055] In some feasible implementations, the loudness decay curve can be determined based on the target sound source type corresponding to the target sound source, combined with the mapping relationship between different sound source types and loudness decay curves.

[0056] Here, a mapping relationship between different sound source types and loudness decay curves can be established in advance. When the target audio generated by the target sound source is triggered to play in the virtual scene, the electronic device can determine the loudness decay curve corresponding to the target sound source type in the pre-established mapping relationship according to the target sound source type corresponding to the target sound source.

[0057] For example, wind sound is a type of sound source. When it is necessary to play wind sound in a virtual scene, the loudness attenuation curve corresponding to the wind sound is determined in a pre-established mapping relationship by using wind sound as the target sound source type.

[0058] It's worth noting that in the above mapping relationship, the attenuation curves for each sound source type show that in the first distance interval, the loudness of the audio produced by the target sound source is positively correlated with distance, while in the second distance interval, the loudness is negatively correlated with distance. Furthermore, the loudness at the beginning of the second distance interval is the same as the loudness at the end of the first distance interval. The main difference between the various loudness attenuation curves lies in the loudness values ​​corresponding to different distances. Therefore, it can be understood that the various loudness attenuation curves actually use different loudness values ​​to simulate the sound effect of different sound source types in the virtual scene, mimicking the real-world sound of that sound source type.

[0059] Therefore, by determining the loudness decay curve by combining the target sound source type corresponding to the target sound source with the mapping relationship between different sound source types and loudness decay curves, the listening effect of the audio of that sound source type in the real world can be accurately simulated in the virtual scene, so as to make the virtual scene more realistic.

[0060] In some feasible implementations, in step 130, the target audio can be spatialized according to the orientation and / or distance between the virtual object and the target sound source to obtain the spatialized target audio, and the target loudness can be used as the loudness corresponding to the spatialized target audio, and then the spatialized target audio can be played.

[0061] Here, the electronic device determines the orientation and / or distance between the virtual object and the target sound source based on the pose information of the virtual object and the target sound source in the virtual scene. Then, it performs spatial processing on the target audio based on the orientation and / or distance between the virtual object and the target sound source.

[0062] It should be understood that spatialization processing essentially involves rendering target audio, which lacks orientation and / or distance information, into a virtual scene. For example, spatialization processing of target audio includes distance processing and / or orientation processing, where distance processing alters one or more of the target audio's loudness, frequency, diffusion, and focus based on the distance parameter of the virtual object relative to the target sound source. Orientation processing alters the timbre of the target audio based on the orientation parameter of the virtual object relative to the target sound source.

[0063] It's worth noting that the spatialized target audio obtained by spatializing the target audio is actually an Ambisonic format audio. Ambisonic format is an audio format that represents sound based on spatial location. Ambisonic audio is isotropic, treating sounds from any direction equally.

[0064] It should be noted that the spatialized target audio actually has loudness. Therefore, the original loudness of the spatialized target audio needs to be replaced with the target loudness. That is, the actual loudness of the spatialized target audio is the target loudness. Alternatively, the loudness of the target audio can be adjusted to the target loudness during the spatialization process, thus obtaining a spatialized target audio with the target loudness.

[0065] Then, the electronic device plays back the spatialized target audio through an audio playback system, so that the spatialized target audio can be played through an audio output device connected to the electronic device. At this time, the loudness of the target audio generated by the target sound source presented to the player's auditory system is the target loudness.

[0066] Therefore, by spatializing the target audio, obtaining the spatialized target audio, and using the target loudness as the loudness corresponding to the spatialized target audio, and then playing the spatialized target audio, the target audio played in the virtual scene can be made more realistic, thereby making the virtual scene more realistic and bringing players a more realistic gaming experience.

[0067] In some feasible implementations, the electronic device may also respond to an instruction to play a first type of ambient audio in a virtual scene by acquiring a first original audio signal corresponding to the first type of ambient audio and playing the first original audio signal.

[0068] Here, the audio played in the virtual scene can also include environmental audio belonging to the first type that is triggered to play in the virtual scene.

[0069] The first type of ambient audio refers to the background sound of the virtual scene. It should be understood that the first type of ambient audio can be considered the basic background audio in the virtual scene. This type of ambient audio can be used to create the atmosphere of the virtual scene. Different virtual scenes can have different ambient audio used to create this atmosphere. For example, when the virtual scene is a natural environment, the first type of ambient audio could be bird calls, insect chirps, etc., used to create this natural atmosphere. It should be noted that the first type of ambient audio does not necessarily have a corresponding concrete virtual object in the virtual scene. That is, players can hear the first type of ambient audio in various areas of the virtual scene, not just when a player-controlled virtual object appears in a particular scene. For example, bird calls are not triggered only when a virtual "bird" appears in the virtual scene.

[0070] When the electronic device receives an instruction to play the first type of environmental audio in the virtual scene, the electronic device acquires the corresponding first original audio signal and plays the first original audio signal directly. That is, in this embodiment of the disclosure, for the first type of environmental audio, the electronic device does not perform any processing on the first type of environmental audio, including spatialization processing, but directly plays the corresponding first original audio signal to create a realistic virtual scene experience.

[0071] It should be understood that the first raw audio signal refers to the unprocessed raw audio signal emitted in the real world, pre-made by the game developers. The audio format of this first raw audio signal can be the 7.1.4 audio format.

[0072] Additionally, it should be noted that the instruction to play type 1 ambient audio in a virtual scene can be triggered when the virtual scene is first displayed.

[0073] Therefore, by directly playing the first original audio signal, the background sound of the virtual scene can be stably represented in the virtual scene, thereby creating a more realistic game atmosphere for the player.

[0074] In some feasible implementations, the electronic device may also respond to an instruction to play a second type of ambient audio in a virtual scene by acquiring a second original audio signal corresponding to the second type of ambient audio, spatializing the second original audio signal according to the orientation between the virtual object and the sound source that generates the second type of ambient audio, obtaining a first target ambient audio, and playing the first target ambient audio.

[0075] Here, the audio played in the virtual scene can also include triggering the playback of ambient audio of type two within the virtual scene. It should be understood that type two ambient audio can be understood as the basic background audio in the virtual scene. Of course, type two ambient audio can be optional ambient sound effects within the virtual scene. When type two ambient audio is configured in the virtual scene, a command to play type two ambient audio in the virtual scene is triggered.

[0076] The second type of ambient audio can be used to create the atmosphere of a virtual scene. Different virtual scenes can have different ambient audio to create that atmosphere. For example, when the virtual scene is a natural environment, the second type of ambient audio could be bird calls, insect chirps, etc., used to create that natural atmosphere. It's important to note that the second type of ambient audio does not necessarily have a corresponding physical virtual object in the virtual scene. That is, players can hear the second type of ambient audio in all areas of the virtual scene, rather than it only appearing when a player-controlled virtual object appears in a particular scene. For example, bird calls are not the first type of ambient audio triggered only when a virtual "bird" appears in the virtual scene.

[0077] When the electronic device receives an instruction to play the second type of environmental audio in the virtual scene, the electronic device acquires the corresponding second original audio signal and performs spatial processing on the second original audio signal according to the orientation between the virtual object and the sound source that generates the second type of environmental audio to obtain the first target environmental audio. Then, the first target environmental audio is played back through the audio playback system so as to play the first target environmental audio through the audio output device connected to the electronic device.

[0078] It should be understood that the second raw audio signal refers to the unprocessed raw audio signal emitted in the real world, pre-made by the game developers. The audio format of this second raw audio signal can be Ambisonic.

[0079] Furthermore, the specific implementation of spatial processing of the second original audio signal based on orientation can be found in the relevant description of the above embodiments, and will not be repeated here.

[0080] The orientation between the virtual object and the sound source that generates the second type of ambient audio can be determined based on the virtual object's posture in the virtual scene and the position of the sound source that generates the second type of ambient audio.

[0081] It is important to note that by spatializing the second original audio signal based on orientation, the output first target environment audio can achieve different sound effects depending on the rotation of the virtual object. For example, if the second original audio signal is characterized by "bird chirping in the player's left ear and dog barking in the player's right ear," when the player rotates the virtual object, the spatially processed first target environment audio will show that "the bird chirping and dog barking will appear in the left or right ear depending on the player's orientation." That is, as the orientation of the virtual object changes, the dog barking, which is fixed in the player's right ear, will also appear in the player's left ear, and the bird chirping, which is fixed in the player's left ear, will also appear in the player's right ear.

[0082] Therefore, by spatializing the second original audio signal according to the orientation between the virtual object and the sound source that generates the second type of environmental audio, the second type of environmental audio played in the virtual scene can be made more realistic, thereby creating a more realistic virtual scene for the player through realistic environmental audio.

[0083] It is worth noting that the first type and the second type of environmental audio in the above embodiments can actually be understood as the same type of environmental audio. For this type of environmental audio, game developers can choose to play either the first original audio signal directly or the processed first target environmental audio and then played, depending on the needs of different virtual scenes. For example, regarding bird calls, for different types of bird calls in different virtual scenes, the developers can choose to play either the first original audio signal or the first target environmental audio to present the bird call as background sound in the virtual scene.

[0084] In some feasible implementations, the electronic device may also respond to an instruction to play a third type of environmental audio in a virtual scene, acquire a third original audio signal corresponding to the third type of environmental audio, and perform spatialization processing on the third original audio signal according to the first position information of the virtual object and the second position information of the scene point sound source that generates the third type of environmental audio to obtain a second target environmental audio, and then play the second target environmental audio.

[0085] Here, the audio played in the virtual scene may also include playing a third type of ambient audio. It should be understood that the second type of ambient audio can be optional ambient audio within the virtual scene. When the virtual scene is configured with a second type of ambient audio, a command to play the second type of ambient audio in the virtual scene is triggered.

[0086] The third type of ambient audio refers to the ambient audio generated by scene point sound sources within the virtual scene. This type of ambient audio is perceived as having a fixed location effect in the user's auditory system. It should be understood that a scene point sound source can be understood as a virtual object present in the virtual scene, and the third type of ambient audio is the ambient sound emitted by that virtual object. For example, the scene point sound source could be an element such as a campfire, fountain, or waterfall set up in the virtual scene. The ambient audio generated by the scene point sound source has a fixed location effect in the user's auditory system; that is, the player can clearly perceive the specific location of the scene point sound source within the virtual scene through the ambient audio generated by the scene point sound source.

[0087] When the electronic device receives an instruction to play a third type of environmental audio in a virtual scene, the electronic device acquires the corresponding third original audio signal and performs spatial processing on the third original audio signal based on the first position information of the virtual object and the second position information of the scene point sound source that generates the third type of environmental audio, to obtain the second target environmental audio. Then, the second target environmental audio is played back through the audio playback system so that it can be played through the audio output device connected to the electronic device.

[0088] It should be understood that the third raw audio signal refers to the unprocessed, raw audio signal emitted by virtual items pre-made by game developers in the real world. This third raw audio signal can be in the Ambisonic audio format.

[0089] Based on the first position information of the virtual object and the second position information of the scene point sound source generating the third type of environmental audio, the third original audio signal is spatialized. In essence, this involves determining the distance between the virtual object and the scene point sound source based on the first position information of the virtual object and the second position information of the scene point sound source in the virtual scene. Then, the third original audio signal is spatially processed according to the determined distance. Distance processing involves changing one or more of the loudness, frequency, diffusion, and focus of the target audio based on the distance between the virtual object and the scene point sound source.

[0090] It is worth noting that the spatialized second target environment audio, when presented to the player's auditory system, has the following effect: the closer the virtual object is to the scene sound source, the louder the second target environment audio emitted by that scene sound source; conversely, the farther the virtual object is from the scene sound source, the lower the loudness of the second target environment audio emitted by that scene sound source.

[0091] Therefore, by spatializing the third original audio signal based on the first position information of the virtual object and the second position information of the scene point sound source that generates the third type of environmental audio, the second target environmental audio can be obtained. This makes the third type of environmental audio played in the virtual scene more realistic, thereby creating a more realistic virtual scene for the player through realistic environmental audio.

[0092] It is worth noting that, in the embodiments of this disclosure, the first type of ambient audio, the second type of ambient audio, the third type of ambient audio, and the target audio generated by the target sound source can be arbitrarily combined. For example, in a virtual scene, the first type of ambient audio, the second type of ambient audio, the third type of ambient audio, and the target audio generated by the target sound source can be played simultaneously, or the first type of ambient audio and the target audio generated by the target sound source can be played simultaneously.

[0093] It should be understood that when multiple audio sources are simultaneously triggered in a virtual scene, including first-type ambient audio, second-type ambient audio, third-type ambient audio, and target audio generated by the target sound source, the electronic device can mix the triggered audio sources and then output the mixed audio. For example, assuming that first-type ambient audio and target audio are triggered simultaneously, the first original audio signal can be mixed with the target audio, and then the mixed audio can be output.

[0094] By using the first type of ambient audio, the second type of ambient audio, the third type of ambient audio, and the target audio generated by the target sound source, the ambient audio played in the virtual scene can be made more layered and realistic, thereby creating a more realistic gaming experience for players.

[0095] Of course, in other implementations, the virtual scene can also play a fourth type of ambient audio, which refers to the background noise of the virtual scene. For example, assuming the virtual scene is a room, the fourth type of ambient audio can be the room's ambient noise. This ambient noise is an environmental sound that a room would inevitably have in the real world and does not change with changes within the room. Exemplarily, the fourth original audio signal corresponding to the fourth type of ambient audio can be played directly. This fourth original audio signal refers to the unprocessed original audio signal emitted in the real world, pre-made by the game developers. The audio format of this fourth original audio signal can be the 7.1.4 audio format.

[0096] Figure 3 This is a schematic diagram of the structure of an audio processing device according to some embodiments. For example... Figure 3 As shown, this disclosure provides an audio processing apparatus 300, which includes:

[0097] The acquisition module 301 is configured to acquire target audio generated by a target sound source in a virtual scene;

[0098] The determining module 302 is configured to determine the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, combined with a loudness attenuation curve. The loudness attenuation curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, and in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance.

[0099] Output module 303 is configured to output the target audio based on the target loudness.

[0100] Optionally, the first distance interval includes a first sub-interval and a second sub-interval, which increase sequentially. In the first sub-interval, the loudness corresponding to different distances is consistent. In the second sub-interval, the loudness of the audio generated by the target sound source is positively correlated with the distance.

[0101] Optionally, the determining module 302 is specifically configured as follows:

[0102] Based on the target sound source type corresponding to the target sound source, and combined with the mapping relationship between different sound source types and loudness attenuation curves, the loudness attenuation curve is determined.

[0103] Optionally, the output module 303 is specifically configured as follows:

[0104] Based on the orientation and / or distance between the virtual object and the target sound source, the target audio is spatialized to obtain the spatialized target audio;

[0105] The target loudness is used as the loudness corresponding to the spatialized target audio, and the spatialized target audio is played.

[0106] Optionally, the audio processing device 300 further includes:

[0107] The first audio acquisition unit is configured to acquire a first original audio signal corresponding to the first type of environmental audio in response to an instruction to play a first type of environmental audio in the virtual scene, wherein the first type of environmental audio is the background sound of the virtual scene;

[0108] The first playback unit is configured to play the first original audio signal.

[0109] Optionally, the audio processing device 300 further includes:

[0110] The second audio acquisition unit is configured to acquire a second original audio signal corresponding to the second type of environmental audio in response to an instruction to play a second type of environmental audio in the virtual scene, wherein the second type of environmental audio is the background sound of the virtual scene;

[0111] The first processing unit is configured to spatialize the second original audio signal according to the orientation between the virtual object and the sound source that generates the second type of ambient audio, so as to obtain the first target ambient audio.

[0112] The second playback unit is configured to play the audio from the first target environment.

[0113] Optionally, the audio processing device 300 further includes:

[0114] The third audio acquisition unit is configured to acquire a third original audio signal corresponding to the third type of environmental audio in response to an instruction to play a third type of environmental audio in the virtual scene, wherein the third type of environmental audio is environmental audio generated by a scene point sound source in the virtual scene.

[0115] The second processing unit is configured to perform spatial processing on the third original audio signal based on the first location information of the virtual object and the second location information of the scene point sound source that generates the third type of environmental audio, so as to obtain the second target environmental audio.

[0116] The third playback unit is configured to play the audio from the second target environment.

[0117] The functional logic executed by each functional module in the aforementioned audio processing device 300 has been described in detail in the section on methods, and will not be repeated here.

[0118] The following is for reference. Figure 4 The diagram illustrates a structural schematic of an electronic device (e.g., a terminal device or a server) 600 suitable for implementing embodiments of the present disclosure. The terminal device in the embodiments of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0119] like Figure 4As shown, electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 601, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 602 or a program loaded from storage device 608 into random access memory (RAM) 603. RAM 603 also stores various programs and data required for the operation of electronic device 600. Processing device 601, ROM 602, and RAM 603 are interconnected via bus 604. Input / output (I / O) interface 605 is also connected to bus 604.

[0120] Typically, the following devices can be connected to I / O interface 605: input devices 606 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 607 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 608 including, for example, magnetic tapes, hard disks, etc.; and communication devices 609. Communication device 609 allows electronic device 600 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 An electronic device 600 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0121] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, it performs the functions defined in the methods of embodiments of this disclosure.

[0122] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0123] In some implementations, electronic devices can communicate using any currently known or future-developed network protocol, such as HTTP (Hypertext Transfer Protocol), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0124] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0125] The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to:

[0126] Obtain the target audio generated by a target sound source in a virtual scene; determine the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, combined with a loudness decay curve, wherein the loudness decay curve includes a first distance interval and a second distance interval, the first distance interval and the second distance interval increasing sequentially, in the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with distance, in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with distance; output the target audio based on the target loudness.

[0127] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including but not limited to object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0128] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0129] The modules described in the embodiments of this disclosure can be implemented in software or hardware. The names of the modules are not, in some cases, intended to limit the functionality of the module itself.

[0130] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0131] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0132] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0133] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0134] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative forms of implementing the claims. Regarding the apparatus in the above embodiments, the specific manner in which the various modules perform their operations has been described in detail in the embodiments relating to the method, and will not be elaborated upon here.

Claims

1. An audio processing method, characterized in that, include: Acquire the target audio generated by the target sound source in the virtual scene; Based on the target distance between the target sound source and the virtual object, and combined with the loudness decay curve, the target loudness corresponding to the target audio is determined. The loudness decay curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, and in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance. The target audio is output based on the target loudness.

2. The method according to claim 1, characterized in that, The first distance interval includes a first sub-interval and a second sub-interval, which increase sequentially. In the first sub-interval, the loudness corresponding to different distances is consistent. In the second sub-interval, the loudness of the audio generated by the target sound source is positively correlated with the distance.

3. The method according to claim 1, characterized in that, The loudness decay curve is determined through the following steps: Based on the target sound source type corresponding to the target sound source, and combined with the mapping relationship between different sound source types and loudness attenuation curves, the loudness attenuation curve is determined.

4. The method according to claim 1, characterized in that, The step of outputting the target audio based on the target loudness includes: Based on the orientation and / or distance between the virtual object and the target sound source, the target audio is spatialized to obtain the spatialized target audio; The target loudness is used as the loudness corresponding to the spatialized target audio, and the spatialized target audio is played.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to an instruction to play a first type of ambient audio in the virtual scene, a first original audio signal corresponding to the first type of ambient audio is obtained, wherein the first type of ambient audio is the background sound of the virtual scene; Play the first original audio signal.

6. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to an instruction to play a second type of ambient audio in the virtual scene, a second original audio signal corresponding to the second type of ambient audio is obtained, wherein the second type of ambient audio is the background sound of the virtual scene; Based on the orientation between the virtual object and the sound source that generates the second type of environmental audio, the second original audio signal is spatialized to obtain the first target environmental audio. Play the audio from the first target environment.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: In response to an instruction to play a third type of ambient audio in the virtual scene, a third original audio signal corresponding to the third type of ambient audio is obtained, wherein the third type of ambient audio is ambient audio generated by a scene point sound source in the virtual scene; Based on the first location information of the virtual object and the second location information of the scene point sound source that generates the third type of environmental audio, the third original audio signal is spatialized to obtain the second target environmental audio. Play the audio from the second target environment.

8. An audio processing apparatus, characterized in that, include: The acquisition module is configured to acquire target audio generated by a target sound source in a virtual scene. The determination module is configured to determine the target loudness corresponding to the target audio based on the target distance between the target sound source and the virtual object, combined with a loudness attenuation curve. The loudness attenuation curve includes a first distance interval and a second distance interval, which increase sequentially. In the first distance interval, the loudness of the audio generated by the target sound source is positively correlated with the distance, and in the second distance interval, the loudness of the audio generated by the target sound source is negatively correlated with the distance. The output module is configured to output the target audio based on the target loudness.

9. A computer-readable medium having a computer program stored thereon, characterized in that, When executed by the processing device, the program implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device, characterized in that, include: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the steps of the method according to any one of claims 1 to 7.

Images