Sound effect processing method for virtual scene

Abstract

The invention provides a sound effect processing method for a virtual scene. The sound effect processing method for the virtual scene comprises the following steps: acquiring at least one piece of sound source parameter information for a target virtual scene; determining a sound source sound effect rule corresponding to each piece of sound source parameter information in a preset rule mapping library, and determining at least one piece of sound source sound effect information according to each piece of sound source sound effect rule and each piece of sound source parameter information; and synthesizing scene sound effect information of the target virtual scene according to the sound effect information of each sound source. According to the method, the sound effect is dynamically synthesized in real time through the multiple pieces of sound source parameter information in the target virtual scene, and the dependence on a fixed sound effect library is eliminated. Meanwhile, the multiple pieces of sound source sound effect information are generated through the multiple pieces of sound source parameter information, the multiple pieces of sound source sound effect information are fused, the types of the sound effect information are enriched, and the use experience of the user is improved.

Description

Technical Field

[0001] This application relates to the field of computer technology, and particularly to a method for processing sound effects in a virtual scene. This application also relates to a sound effect processing device for a virtual scene, a computing device, a computer-readable storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, the development of software has been driven, which in turn has promoted the progress of the game industry. In recent years, various games have emerged in an endless stream, and the competition in the game industry has become increasingly fierce.

[0003] In various games, the effect of game audio is an important component. Most of the current game sound effect systems have problems such as static sound effects being repeated, unable to dynamically adapt to the game scene, unable to provide personalized settings for users, and unable to be adjusted according to user preferences. Therefore, there is an urgent need for a new sound effect processing method to solve the above problems. Summary of the Invention

[0004] In view of this, the embodiments of this application provide a method for processing sound effects in a virtual scene. This application also relates to a sound effect processing device for a virtual scene, a computing device, a computer-readable storage medium, and a computer program product to solve the above problems existing in the prior art.

[0005] According to the first aspect of the embodiments of this application, a method for processing sound effects in a virtual scene is provided, including: Obtaining at least one sound source parameter information for a target virtual scene; Determining the sound source sound effect rules corresponding to each sound source parameter information in a preset rule mapping library, and determining at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information; Synthesizing the scene sound effect information of the target virtual scene according to each sound source sound effect information.

[0006] According to the second aspect of the embodiments of this application, a sound effect processing device for a virtual scene is provided, including: An obtaining module configured to obtain at least one sound source parameter information for a target virtual scene; A determining module configured to determine the sound source sound effect rules corresponding to each sound source parameter information in a preset rule mapping library, and determine at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information; A synthesizing module configured to synthesize the scene sound effect information of the target virtual scene according to each sound source sound effect information.

[0007] According to the third aspect of the embodiments of this application, a method for processing sound effects in a virtual scene is provided, including: Obtain user feature information and at least one sound source parameter information for a target virtual scene; Input the user feature information and each sound source parameter information into a sound effect synthesis model to obtain the scene sound effect information corresponding to the target virtual scene output by the sound effect synthesis model, where the scene sound effect information is generated according to user reference sound effect information and at least one sound source sound effect information, the user reference sound effect information is generated according to the user feature information, and the at least one sound source sound effect information is generated according to the at least one sound source parameter information.

[0008] According to a fourth aspect of the embodiments of the present application, there is provided a computing device, including: A memory and a processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0009] According to a fifth aspect of the embodiments of the present application, there is provided a computer-readable storage medium storing computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0010] According to a sixth aspect of the embodiments of the present application, there is provided a computer program product including computer programs / instructions, and when the computer programs / instructions are executed by a processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0011] The sound effect processing method for the virtual scene provided by the present application obtains at least one sound source parameter information for a target virtual scene; determines the sound source sound effect rules corresponding to each sound source parameter information in a preset rule mapping library, and determines at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information; synthesizes the scene sound effect information of the target virtual scene according to each sound source sound effect information.

[0012] An embodiment of the present application realizes the real-time dynamic synthesis of sound effects through multiple sound source parameter information in a target virtual scene, getting rid of the dependence on a fixed sound effect library. At the same time, multiple sound source parameter information generates multiple sound source sound effect information, and the fusion between multiple sound source sound effect information enriches the types of sound effect information and improves the user experience. Description of the Drawings

[0013] Figure 1 is a flowchart of a sound effect processing method for a virtual scene provided by an embodiment of the present application; Figure 2 is a processing flowchart of a sound effect processing method for a virtual scene applied to a game scene provided by an embodiment of the present application; Figure 3 It is a flowchart of a method for processing sound effects in a virtual scene provided by another embodiment of the present application; Figure 4 It is a schematic structural diagram of a device for processing sound effects in a virtual scene provided by an embodiment of the present application; Figure 5 It is a structural block diagram of a computing device provided by an embodiment of the present application. Detailed implementation manners

[0014] Many specific details are set forth in the following description in order to provide a thorough understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application. Therefore, the present application is not limited by the specific implementations disclosed below.

[0015] The terms used in one or more embodiments of the present application are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of the present application. The singular forms "a", "the", and "said" used in one or more embodiments of the present application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present application refers to and includes any or all possible combinations of one or more of the associated listed items.

[0016] It should be understood that although the terms first, second, etc. may be used in one or more embodiments of the present application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".

[0017] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use, and processing of relevant data need to comply with the relevant laws, regulations, and standards in the relevant regions, and corresponding operation entrances are provided for the user to select authorization or rejection.

[0018] With the development of computer technology, it has driven the development of software development, and then promoted the progress of the game industry. In recent years, various games have emerged in an endless stream, and the competition in the game industry has become increasingly fierce.

[0019] In various games, the effect of game audio is a very important component. Most current game sound effect systems have the following limitations: 1. Static sound effect repetition: Relying on a fixed sound effect library leads to repeated use of materials, and players are prone to aesthetic fatigue.

[0020] 2. Insufficient dynamic adaptation: Sound effects are based on simple logic (such as scene switching) and cannot dynamically adjust details according to multi-dimensional parameters.

[0021] 3. Limited user control: The current game sound effect system only provides basic volume adjustment functions and cannot deeply meet the personalized needs of users. Based on this, in this application, a method for processing sound effects in a virtual scene is provided. This application also relates to a device for processing sound effects in a virtual scene, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail one by one in the following embodiments.

[0022] Figure 1 The flowchart of a method for processing sound effects in a virtual scene according to an embodiment of this application is shown, which specifically includes the following steps: Step 102: Obtain at least one sound source parameter information for the target virtual scene.

[0023] Among them, in the method provided in this application, the target virtual scene can be understood as the scene where sound effects need to be generated. A virtual scene is a scene virtually generated by computer technology means. For example, in a game scene, the scene where the character operated by the user is located is the target virtual scene; for another example, in an animation scene, the scene shown to the user is the target virtual scene.

[0024] In the method provided in the embodiment of this application, the user can render and generate the target virtual scene in the terminal by browsing or operating the terminal. The purpose of the method provided in the embodiment of this application is to dynamically generate corresponding sound effects for the target virtual scene. For example, taking a game scene as an example, when the character operated by the user is in a certain game scene, corresponding game sound effects can be generated according to different operations of the user, so that the user's gameplay is more immersive and the user's game experience is improved.

[0025] In the method for processing sound effects in a virtual scene provided in the embodiment of this application, it is applied to the terminal running the target virtual scene. For example, if the user uses the terminal to generate the target virtual scene, this method is applied to the terminal; for another example, if the user uses cloud services and the target virtual scene is generated in the cloud services and transmitted to the user terminal for display, this method is applied to the cloud server deploying the cloud services. In the method provided in the embodiment of this application, the execution entity running this method is not limited and is subject to actual applications.

[0026] The sound source parameter information can be understood as the parameter information used to generate the scene sound effects in the target virtual scene. In practical applications, there may be multiple scene factors in the target virtual scene, and the factors used to generate the corresponding scene sound effects of the target virtual scene can all be understood as the sound source parameter information. In the method provided in the embodiments of the present application, obtaining at least one piece of sound source parameter information for the target virtual scene includes: Obtaining at least one of the environmental parameter information, character behavior information, and user configuration information for the target virtual scene.

[0027] Among them, the environmental parameter information can be understood as the environmental information in the target virtual scene, such as terrain type, weather condition, day and night time, etc. The character behavior information can be understood as the behavior information of the virtual character in the target virtual scene, such as moving speed, action frequency, character status, etc. The user configuration information can be understood as the personalized sound effect configuration pre-configured by the user for the target virtual scene, such as high-frequency, medium-frequency, low-frequency and other configuration information.

[0028] In a specific implementation manner provided in the present application, taking a game scene as an example, the target virtual scene at this time is a game scene, that is, the user operates the game character to attack other characters in the game. At this time, the environmental parameter information (such as terrain type, weather condition), character behavior information (such as character moving speed, attack frequency, jumping height, being attacked status, etc.), and user configuration information (frequency response curve, such as 5-band EQ: low frequency 80Hz ± 15dB, medium frequency 1kHz ± 3dB, etc.) in the target game scene can be obtained.

[0029] In practical applications, the sound source parameter information can be any one of the environmental parameter information, character behavior information, and user configuration information. Further, in order to better dynamically generate the scene sound effects of the target virtual scene, the above three types of information can be obtained. And in practical applications, there may be no character behavior information. For example, the user does not operate the game character to have corresponding behaviors in the target virtual scene, so the character behavior information may be empty; or there may also be no user configuration information. For example, the user does not understand the configuration parameters of the sound effects, so no corresponding configuration is made, so the user configuration information may be empty. It should be noted that in the preferred technical solution provided in the embodiments of the present application, the environmental parameter information, character behavior information, and user configuration information of the target virtual scene are obtained, and there is a possibility that these information are empty. In this method, the above information needs to be obtained, and if there is no specific information content, it can be set to be empty or 0.

[0030] In the method provided in the embodiments of the present application, obtaining the environmental parameter information for the target virtual scene includes: Obtain the environmental parameter information of the target virtual scene according to the virtual scene engine corresponding to the target virtual scene; Obtain the character behavior information for the target virtual scene, including: Obtain the character behavior information of the target virtual scene according to the character controller corresponding to the target virtual scene; Obtain the user configuration information for the target virtual scene, including: Provide a user configuration page, where the user configuration page includes sound effect configuration parameter information for the target virtual scene; In response to the operation information for the user configuration interface, obtain the user configuration information for the target virtual scene.

[0031] In the method provided in the embodiments of the present application, the target virtual scene is rendered and generated by the virtual scene engine. Therefore, the environmental parameter information corresponding to the target virtual scene can be obtained through the virtual scene engine. For example, taking a game scene as an example, the game engine renders and generates the surface information, and the surface material type in the game scene can be obtained in real time through the terrain gateway data in the game engine (such as UnityTerrain.heightmap or Unreal Landscape component). The environmental scene information in the game scene is generated by the game engine according to the rendering information. Therefore, the environmental parameter information in the game scene can be obtained through the game engine.

[0032] For example, when performing terrain type recognition, the surface material type (such as rock, water surface, grassland) is obtained in real time through the game engine, and then through the material index mapping table, the obtained information is recorded in the form of a bitmap (determine the corresponding surface material according to flag 1). Assuming that the surface material types in the game engine include rock, water surface, grassland, then "010" can be used to represent that the current surface material type is water surface, where the first 0 corresponds to rock, the second 1 represents water surface, and the third 0 represents operation. Similarly, after terrain type recognition, if it is determined that the current surface material is rock, it can be identified by "100".

[0033] In practical applications, the precipitation intensity value of the target game scene can also be obtained according to the game meteorological system (usually represented by a range of 0-1), mapped to precipitation intensity parameters, and the wind sound frequency band is dynamically adjusted through fast Fourier transform analysis (such as activating the 1-3kHz frequency band of wind sound during heavy rain).

[0034] In the method provided by this application, the role behavior information of the target virtual scene is obtained through a character controller. For example, in a game scene, the instantaneous speed of the game character controlled by the user can be obtained through a character controller. Specifically, sampling can be performed every preset time (such as once every 0.1 seconds) to trigger the footstep frequency (footstep frequency = movement speed / step length, and the step length can be determined according to the character information). In a game scene, the role behavior information can also include the attack frequency statistics, that is, the number of attack events within a unit time (such as attacks per second, Attack PerSecond, APM). Smoothing processing is performed through the sliding window algorithm to trigger the dynamic adjustment of the sound effect parameters (such as activating the combo sound effect enhancement when the attack interval < 0.3 seconds).

[0035] In the method provided by this application, a user configuration interface can also be provided for the user. The user configuration page provides the user with a sound effect configuration function in the form of a visual interface. The user can set and adjust the provided sound effect configuration parameters through the user configuration page, so as to generate the corresponding user configuration information of the user. For example, the user can define the frequency response curve in the user configuration interface. The terminal obtains the frequency response curve.

[0036] In the method provided by the embodiments of this application, for at least one sound source parameter information of the target virtual scene, specifically, the environmental parameter information, role behavior information, and user configuration information in the target virtual scene are obtained, so that the sound effect system can better generate the corresponding scene sound effects according to the actual information of the target virtual scene. The acquisition channels of various sound source parameter information are clear, and the data acquisition is convenient, which is beneficial to providing data support for subsequent sound effect synthesis.

[0037] Step 104: Determine the sound source sound effect rules corresponding to each sound source parameter information in the preset rule mapping library, and determine at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information.

[0038] Among them, the preset rule mapping library is a rule library composed of multiple pre-created sound source sound effect rules. The preset sound effect generation and adjustment rules are stored in the preset rule mapping library, and the actual scene requirements of the target virtual scene can be matched according to the priority algorithm. The sound source sound effect rule can be understood as the rule stored in the preset rule mapping library. In practical applications, each sound source parameter information can correspond to one or more sound source sound effect rules. The sound source sound effect information can be understood as the sound effect information generated after each sound source parameter information is processed according to its corresponding sound source sound effect rule. For example, after the environmental parameter information is processed by the environmental sound effect rule, the environmental sound effect information is obtained; after the role behavior information is processed by the role sound effect rule, the role sound effect information is obtained; after the user configuration information is processed by the user configuration rule, the configuration sound effect information is obtained.

[0039] In the method provided by the embodiments of the present application, after obtaining at least one sound source parameter information in the above steps, the sound source sound effect rules corresponding to each sound source parameter information can be determined in the preset rule mapping library. For example, when the environmental sound effect information includes that the ground is rock, the rock echo algorithm corresponding to the rock can be found from the preset rule mapping library, and the echo attenuation information of the ground can be calculated through the rock echo algorithm. Another example is that when the environmental sound effect includes that the ground is water surface, the water surface reflection rule corresponding to the water surface can be obtained, and the reverberation information of the footsteps on the water surface can be calculated through the water surface reflection rule, and so on.

[0040] In practical applications, the corresponding sound source sound effect rules will be pre-configured for various sound source parameter information. After determining the sound source parameter information, the sound source sound effect rules corresponding to each sound source parameter information can be determined through the preset rule mapping library. And the sound source sound effect information is determined through the sound source sound effect rules and the sound source parameter information.

[0041] In a specific embodiment provided by the present application, the sound source parameter information includes at least one of environmental parameter information, character behavior information, and user configuration information; Determining the sound source sound effect rules corresponding to each sound source parameter information in the preset rule mapping library includes: Analyzing the environmental parameter information to obtain at least one environmental parameter value, and determining the environmental sound effect rules corresponding to each environmental parameter value in the preset rule mapping library according to each environmental parameter value; Analyzing the character behavior information to obtain at least one character behavior value, and determining the character sound effect rules corresponding to each character behavior value in the preset rule mapping library according to each character behavior value; Analyzing the user configuration information to obtain at least one user configuration value, and determining the user configuration rules corresponding to each user configuration value in the preset rule mapping library according to each user configuration value.

[0042] In this embodiment, when the sound source parameter information includes environmental parameter information, the environmental parameter information can be analyzed to obtain at least one environmental parameter value, and the environmental sound effect rules corresponding to each environmental parameter value can be determined in the preset rule mapping library through the environmental parameter values. Among them, the environmental parameter value is the environmental information determined in the environmental analysis parameters. For example, taking a game scene as an example, analyzing the environmental parameter information includes that the terrain is water surface and it is raining, then the environmental parameter values are the water depth coefficient and the rain sound intensity parameter, and the environmental sound effect rules include the water surface rule and the rain rule.

[0043] When the sound source parameter information includes character behavior information, at least one character behavior value can be obtained by parsing the character behavior information, and the character sound effect rules corresponding to each character behavior value can be determined from the preset rule mapping library. The character behavior value is the behavior information of the character in the target virtual scene. For example, still taking the game scene as an example, parsing the character behavior information includes that the character is attacking and jumping at the same time, then the character behavior values are the attack speed and the jumping height, and the character sound effect rules include the attack speed sound effect rule and the jumping landing sound intensity rule.

[0044] When the sound source parameter information includes user configuration information, parse the user configuration information to obtain the user configuration value. For example, taking the frequency response curve defined by the user as an example, after obtaining the user-defined frequency response curve, it can be converted into filter parameters and stored as a corresponding structure for subsequent real-time calls.

[0045] In practical applications, after obtaining the sound source sound effect rules, the sound source sound effect information can be further determined according to the sound source parameter information. Specifically, in a specific implementation manner provided in this application, at least one sound source sound effect information is determined according to each sound source sound effect rule and each sound source parameter information, including: Determine the target sound source sound effect rule and the target sound source parameter information corresponding to the target sound source sound effect rule, where the target sound source sound effect rule is any one of the sound source sound effect rules; Extract the target sound source sound effect formula in the target sound source sound effect rule; Determine the target sound source sound effect information corresponding to the target sound source sound effect rule according to the target sound source sound effect formula and the target sound source parameter information.

[0046] In practical applications, there are usually multiple sound source sound effect rules, and the processing methods for each sound source sound effect rule are the same. In this embodiment, one of the sound source sound effect rules can be used as an example for explanation. Based on this, determine the target sound source sound effect rule, and the target sound source sound effect rule can be any one of the sound source sound effect rules.

[0047] When determining the target sound source sound effect rule, the target sound source parameter information corresponding to the target sound source sound effect rule can also be determined. Determine the target sound source sound effect formula from the target sound source sound effect rule, and determine the target sound source sound effect information corresponding to the target sound source sound effect rule according to the target sound source sound effect formula and the target sound source parameter information.

[0048] The target sound source sound effect formula can be understood as the specific rule in the target sound source sound effect rule. Substitute the target sound source parameter information into the target sound source sound effect formula, and the corresponding target sound source sound effect information can be calculated.

[0049] For example, taking the environmental sound effect rule as an example, the target sound source sound effect rule is the water surface reflection rule, and its corresponding target sound source parameter information is the water depth coefficient. The target sound source sound effect formula in the water surface reflection rule is "reverberation time T = base value * water depth coefficient", where the base value is the base value of the footsteps, and the water depth coefficient is the water depth coefficient of the current water surface. The water depth coefficient is usually 0.5 - 2.

[0050] Another example, taking the character sound effect rule as an example, the target sound source sound effect rule is the attack speed rule, and its corresponding target sound source parameter information is the attack speed. The target sound source sound effect formula in the attack speed rule is "semitone offset = base value + 0.2 * (attack speed - 1)", where the semitone offset refers to the pitch offset, and the base value is the base value of the pitch offset. Obtain the attack speed (times / second) of the character, and substitute the attack speed and the base value into the above formula to calculate the semitone offset.

[0051] Another example, still taking the character sound effect rule as an example, the target sound source sound effect rule is the jump landing rule, and its corresponding target sound source parameter information is the jump height (unit: meter). The target sound source sound effect formula in the jump landing rule is "landing sound amplitude = sqrt(h) * material hardness coefficient, where the material hardness coefficient is related to the ground material, rock = 1, grass = 0.3", where sqrt is the square root. Substitute the jump height of the character and the coefficient corresponding to the ground material into the formula to obtain the landing sound amplitude corresponding to the target virtual scene.

[0052] Through the above steps, the sound source sound effect information corresponding to each sound source parameter information can be obtained according to each sound source sound effect rule. The sound source sound effect information is used to generate the scene sound effect information of the target virtual scene in the subsequent process. The sound source sound effect information is an important part of the final scene sound effect information.

[0053] Step 106: Synthesize the scene sound effect information of the target virtual scene according to each sound source sound effect information.

[0054] In the method provided in the embodiment of the present application, after obtaining each sound source sound effect information, each sound source sound effect information can be synthesized to generate the scene sound effect information of the target virtual scene. Among them, in a target virtual scene, there may be multiple sound source information, and each sound source sound effect information is the sound effect information of a single sound source in the target virtual scene. The scene sound effect information can be understood as the sound effect information obtained by fusing each sound source sound effect information.

[0055] In the method provided by the embodiments of the present application, corresponding sound source parameter weights can be assigned to each sound source sound effect information according to a preset weighted scoring model, and then each sound source sound effect information is fused based on the sound source parameter weights to obtain scene sound effect information. In a specific implementation manner provided by the present application, synthesizing the scene sound effect information of the target virtual scene according to each sound source sound effect information includes: Determining the sound source parameter weights corresponding to each sound source sound effect information according to the sound source parameter weights corresponding to each sound source parameter information; Synthesizing the scene sound effect information of the target virtual scene according to each sound source sound effect information and each sound source parameter weight.

[0056] Among them, the sound source parameter weight can be understood as the weight information corresponding to each sound source parameter information preset through a weighted scoring model in the method provided by the present application. For example, in the above steps, it is determined that the sound source parameter information includes environmental parameter information, character behavior information, and user configuration information. Through the weighted scoring model, the weight of the environmental parameter information can be determined to be 50%, the weight of the character behavior information is 30%, and the weight of the user configuration information is 20%.

[0057] After determining the sound source parameter weights corresponding to each sound source parameter information, the sound source parameter weights corresponding to each sound source sound effect information can be further determined. For example, the environmental parameter weight of the environmental sound effect information corresponding to the environmental parameter information is 50%, the character parameter weight of the character sound effect information corresponding to the character behavior information is 30%, and the configuration parameter weight of the configuration sound effect information corresponding to the user configuration information is 20%.

[0058] After determining the sound source parameter weights corresponding to each sound source sound effect information, each sound source sound effect information and each sound source parameter weight can be fused to generate the scene sound effect information of the target virtual scene.

[0059] In a specific implementation manner provided by the present application, if two sound source sound effect information conflict with each other, this weight is also used for calculation to confirm the final parameters. For example, when the user turns down the low-frequency setting, but the environmental sound effect information in the target virtual scene needs to enhance the footsteps, the final parameters are calculated according to the above weights, that is, the final low-frequency gain = (environmental parameter weight * environmental parameter information) + (configuration parameter weight * user configuration information).

[0060] In practical applications, each sound source sound effect information is input into a hierarchical sound effect synthesis engine, and real-time sound effects are superimposed in the sound effect synthesis engine according to each sound source parameter weight, ensuring detailed dynamic adjustment and low-latency output.

[0061] Specifically, the layered sound effect synthesis engine includes a basic layer, an interaction layer, and a feedback layer. In the basic layer, environmental sound effect information is synthesized, and environmental sound effects are generated based on a physical acoustics model. For example, for rain sound synthesis, white noise generators are superimposed with band-pass filters (frequency band 200 - 2000 Hz), and the gain is adjusted according to the rain intensity: rain sound gain = base value + precipitation intensity * 10 dB. For terrain attenuation, the acoustic wave propagation loss is calculated based on distance and material.

[0062] In the interaction layer, the action sound effects of the character are synthesized. For example, the high-frequency ratio is adjusted according to the weapon attack power: high-frequency gain ratio = base value * (1 + attack power / 100). When the player's movement speed is greater than the preset speed threshold, the footstep interval is compressed to 70% of the original rate (achieved through audio clip time scaling), and so on.

[0063] In the feedback layer, through real-time loading of the equalizer, the frequency response curve configured by the user is converted into filter parameters and applied frame by frame before audio output.

[0064] In a specific embodiment provided by the present application, the scene sound effect information of the target virtual scene is synthesized according to each sound source sound effect information and each sound source parameter weight, including: Calculating the sum of weights of the sound source parameter weights corresponding to each sound source sound effect information; When the sum of weights is greater than the preset weight threshold, the scene sound effect information of the target virtual scene is synthesized according to each sound source sound effect information and each sound source parameter weight; When the sum of weights is less than or equal to the preset weight threshold, the scene sound effect information of the target virtual scene is synthesized according to each sound source sound effect information.

[0065] In this method, although corresponding sound source parameter weights have been configured for each sound source sound effect information, in actual applications, some sound source sound effect information may not exist. For example, in a game scene, if the character operated by the user does not move, the character behavior information is empty. Another example is that if the user does not make any configuration, the user configuration information is empty. Therefore, in the method provided by the present application, it is also necessary to further calculate the sum of weights of each sound source sound effect information. If the sum of weights is greater than the preset weight threshold, the scene sound effect information is generated according to the sound source parameter weights. If the sum of weights is less than or equal to the preset weight threshold, and then using the sound source parameter weights for synthesis will result in a poor sound effect. At this time, directly fuse each sound source sound effect information without combining the sound source parameter weights corresponding to each sound source sound effect information.

[0066] For example, still taking the weight of environmental parameter information as 50%, the weight of character behavior information as 30%, the weight of user configuration information as 20%, and the preset weight threshold as 70% as an example. If there is no character behavior information in the target virtual scenario, the sum of the weights of the environmental parameter information and the user configuration information is 70%, which is the same as the preset weight threshold. At this time, there is no need to fuse based on the weights of each sound source parameter, and only the environmental sound effect information and the configuration sound effect information need to be fused.

[0067] In the method provided in the embodiment of the present application, the method further includes: Receiving sound effect feedback information for the scene sound effect information; Determining the sound source sound effect rule to be adjusted according to the sound effect feedback information, and updating the sound source sound effect rule to be adjusted based on the sound effect feedback information.

[0068] In the method provided in the embodiment of the present application, a feedback management module is further provided. Based on the feedback management module, the sound effect feedback information of the user for the scene sound effect information is received, and the sound source sound effect rule to be adjusted is determined in the preset rule mapping library through the sound effect feedback information, and then the sound source sound effect rule to be adjusted is adjusted and updated according to the sound effect feedback information. Through the sound effect feedback information, the sound source sound effect rule can be adjusted in time, so as to generate a sound effect rule that is more suitable for the user.

[0069] In a specific implementation manner provided by the present application, the sound effect feedback information includes user rating information and / or user operation information; Determining the sound source sound effect rule to be adjusted according to the sound effect feedback information, and updating the sound source sound effect rule to be adjusted based on the sound effect feedback information, includes: Determining the sound source sound effect rule to be adjusted and the rule adjustment strategy according to the user rating information, and updating the sound source sound effect rule to be adjusted based on the rule adjustment strategy; When the user operation information meets the rule adjustment condition, determining the sound source sound effect rule to be adjusted and the rule adjustment strategy according to the user operation information, and updating the sound source sound effect rule to be adjusted based on the rule adjustment strategy.

[0070] In this method, the sound effect feedback information specifically includes display feedback information and implicit feedback information. The display feedback information includes user rating information, and the implicit feedback information includes user operation information. The user rating information refers to the rating information of the user for the scene sound effect information, which represents the user's recognition of the scene sound effect information. The user operation information can be understood as the feedback of the operation behavior of the user through the scene sound effect information, so as to judge that the user is not adapted to a certain sound source sound effect rule, and then adjust the sound source sound effect rule.

[0071] Specifically, when the audio effect feedback information is the user rating information, the to-be-adjusted audio effect rule and the rule adjustment strategy corresponding to the user rating information can be obtained. For example, if the user gives a low rating to the scene audio effect information, the rule with a relatively large weight in synthesizing the current scene audio effect information will be selected as the to-be-adjusted sound source audio effect rule, and the rule adjustment strategy will be determined to reduce the weight ratio. If the user's rating is less than the disabled rule rating three times in a row, rule fusing can be triggered to disable the corresponding to-be-adjusted sound source audio effect rule.

[0072] For another example, this method also detects the abnormal behavior of the user (such as switching the character's perspective, etc.) in real time after the scene audio effect information is triggered, so as to determine the to-be-adjusted sound source audio effect rule and the rule adjustment strategy, and then adjust the to-be-adjusted sound source audio effect rule according to the rule adjustment strategy. For example, when the generated scene audio effect information is transmitted to the user's sound source output device, if the character operated by the user frequently rotates the perspective within 10 seconds, the system will determine that the audio effect is overloaded, bringing a bad experience to the user, and automatically reduce the environmental audio effect volume by 10%. In practical applications, the rule weight can be dynamically adjusted through the user operation information. Specifically, weight = weight * (1 - γ) + γ * R, where γ is the learning rate and R is the rule verification score.

[0073] In practical applications, during the process of generating the scene audio effect information, in addition to the above content, the user-related characteristic information can also be further referred to. The user reference audio effect information is generated through the user-related characteristic information, and then the user reference audio effect information is added to the scene audio effect information to enhance the user experience.

[0074] In a specific embodiment provided by this application, it further includes: Obtain the user characteristic information; Generate the user reference audio effect information according to the user characteristic information; Correspondingly, synthesizing the scene audio effect information of the target virtual scene according to each sound source audio effect information includes: Synthesize the scene audio effect information of the target virtual scene according to each sound source audio effect information and the user reference audio effect information.

[0075] Among them, the user characteristic information can be understood as the information related to the user. In practical applications, it can include the user's user environment information, the user's user emotion information, and so on.

[0076] According to the user characteristic information, the user reference audio effect related to the user can be further generated. When generating the scene audio effect information of the target virtual scene, each sound source audio effect information and the user reference audio effect information can be fused to generate the final scene audio effect information.

[0077] Among them, obtaining user feature information includes: obtaining user environment information and / or user emotion information. User environment information can be understood as the environment where the user operates the terminal, where the terminal is used to generate a target virtual scene. For example, taking the target virtual scene as a game scene, when the user operates a mobile phone to control the game in the mobile phone, it is necessary to collect the user environment information of the environment where the user is located.

[0078] Specifically, the environment information can be collected through the terminal used by the user. For example, devices such as IMU (Inertial Measurement Unit, six-axis inertial measurement unit), 3D camera, and environmental sensor array are used to collect the environment information where the user is located. The IMU can collect information such as the acceleration and angular velocity of the terminal; the 3D camera can collect color images and depth images of the surrounding environment; the environmental sensor array can collect information such as temperature, humidity, air pressure, and light in the surrounding environment.

[0079] For the various types of environment information collected, a spatio-temporal alignment mechanism is adopted. The noise in each sensor is eliminated through Kalman filtering, and an environment map is constructed through the SLAM (Simultaneous Localization and Mapping) algorithm. A mapping relationship between "spatial position - environmental parameters" is constructed through the environment map. The environment map can represent a geometric space (such as terrain height, obstacle position, etc.), and can also extract environmental semantic information (such as scenes like forests, urban streets, caves, etc.) and physical specific parameters (such as ground material, obstacle reflectivity, spatial openness, etc.) through the SLAM semantic segmentation module. The environment map is the core hub for generating sound effects based on user information combined with environmental characteristics. Its essence is to convert the discrete data collected by multiple sensors into the same environmental space through spatial modeling.

[0080] After obtaining the user environment information, environmental reference sound effect information can be generated according to the user environment information. Specifically, the user environment information includes an environment map and environmental state data; generating environmental reference sound effect information according to the user environment information includes: Obtaining the user behavior information of the user; Generating environmental sound effect parameters according to the user behavior information and the environmental state information; Generating environmental reference sound effect information according to the environment map and the environmental sound effect parameters.

[0081] Among them, the user environment information includes environmental state data, and the environmental state data can be understood as the above-mentioned environmental semantic information. The environment where the user is located (such as a forest, a cave, an urban street, etc.) can be determined through the environmental state data.

[0082] By inputting the user's user behavior data (such as the user's footsteps rhythm, arm swing frequency, etc.) and environmental state information into the sound effect parameter generation model for analysis, environmental sound effect parameters can be obtained. The sound effect parameter generation model can be a pre-trained deep learning model (such as a Transformer model) that generates sound effect information based on user behavior data and environmental state information. In practical applications, the environmental sound effect parameters include parameter information of the finally generated sound effect, such as pitch, rhythm, timbre, etc.

[0083] After obtaining the environmental sound effect parameters, environmental reference sound effect information can be jointly generated according to the environmental map and the environmental sound effect parameters. Specifically, the environmental sound effect parameters and the constructed environmental map can be input into the generative adversarial network, and the environmental reference sound effect is synthesized in real time through the generative adversarial network. In the generative adversarial network, the user's location information is determined through the environmental map, the rules of the environmental sound effect are determined through the environmental sound effect parameters, and the environmental reference sound effect information matching the environmental map and the environmental sound effect parameters is output through the generative adversarial network.

[0084] In another specific implementation manner provided in this application, obtaining user emotion information includes: Collecting the user's multi-dimensional biometric information, and generating user emotion information according to the multi-dimensional biometric information, where the user emotion information includes basic dimension information and composite dimension information; Correspondingly, generating emotion reference sound effect information according to the user emotion information includes: Determining emotion sound effect parameter information and emotion sound effect fusion mode information corresponding to the user emotion information in a preset emotion parameter database according to the user emotion information.

[0085] In this implementation manner, the user's multi-dimensional biometric information can be collected through a terminal. Specifically, information such as the user's expression can be collected through a front camera on the terminal, and information such as the user's heartbeat and heart rate can be collected through a smart wearable device, etc. The information such as expression, heartbeat, and heart rate is the user's multi-dimensional biometric information.

[0086] After collecting the multi-dimensional biometric information, the feature fusion model can fuse the collected multi-dimensional feature information to obtain a biological joint embedding feature. Using the multi-dimensional feature information can enrich subsequent judgments and reduce misjudgments caused by single-modal features. Specifically, the feature fusion model is trained based on the Transformer architecture, and the semantic alignment, dynamic weight allocation, and long-distance dependence of the multi-dimensional feature information are realized through the self-attention mechanism of the Transformer architecture, generating a biological joint embedding feature for subsequent user emotion discrimination. When the multi-dimensional feature information is fused, it is a process of collaborative fusion among time, semantics, and features, where time alignment is the basis, semantic alignment is the core, and feature collaborative fusion is the goal.

[0087] The generated bio-union embedding features are input into an emotion discrimination model, and the current user emotion information of the user is determined through the emotion discrimination model in combination with the bio-union embedding features. In the method provided in the embodiments of the present application, the user emotion information includes basic dimension information and composite dimension information.

[0088] Among them, the basic dimension information is used to represent the basic emotions of the user, such as "pleasure", "tension", "sadness", "excitement", etc. The composite dimension information is an extended dimension based on the basic dimension information, such as "tension-high concentration", "excitement-low control", etc.

[0089] In practical applications, the emotion discrimination model can also output an emotion intensity value according to the bio-union embedding features. This emotion intensity value is used to determine whether the user's emotion will affect subsequent sound effect changes. For example, when the emotion intensity value is less than a threshold, at this time, the user's emotion information does not need to generate corresponding emotion reference information. The emotion intensity value is used to reduce misjudgment and avoid sudden changes in sound effects caused by noise at the end (such as the player coughing, short facial muscle twitches, etc.).

[0090] After determining the user emotion information, the emotion sound effect parameter information and emotion sound effect fusion mode information corresponding to the user emotion information can be determined in the preset emotion parameter database according to the user emotion information.

[0091] The preset emotion parameter database refers to a sound effect parameter database established in advance for emotion classification, and the sound effect parameters corresponding to each emotion are stored in the preset emotion parameter database. Among them, the preset emotion parameter database includes basic dimension information and composite dimension information; the sound effect parameters include the sound effect parameters corresponding to the basic dimension information and composite dimension information (such as rhythm, pitch, harmony complexity, instrument type, etc.).

[0092] The corresponding emotion sound effect information can be determined in the preset emotion parameter database through the user's emotion information. At the same time, the emotion sound effect fusion mode information of the user can also be determined according to the user emotion information. The emotion sound effect fusion mode information can at least include dynamic mixing, transition, etc. When the user's emotion is linearly transformed, the dynamic mixing mode can be used; when the user's emotion is suddenly changed, the transition mode can be adopted.

[0093] For example, when it is detected through the terminal that the user's heart rate rises and the expression is tense, the music can be gradually accelerated from a lower rhythm to a faster rhythm; if it is detected that the user has a sudden emotional fluctuation (such as a sudden increase in heart rate brought about by a combat scene in a game), the transition technology can be adopted to instantly increase the rhythm and intensity of the sound effect and avoid auditory breaks.

[0094] After obtaining the environmental reference sound effect information and / or emotional reference sound effect information through the above method, user reference sound effect information can be generated based on the environmental reference sound effect information and / or emotional reference sound effect information. That is, in practical applications, the user reference sound effect information can be generated by combining the actual situation of the user and based on the collected information. For example, in a certain virtual scenario, if only the environmental reference sound effect information is collected, the user reference sound effect information can be determined according to the environmental reference sound effect information; if only the emotional reference sound effect information is collected, the emotional reference sound effect information can be determined as the user reference sound effect information; if both the environmental reference sound effect information and the emotional reference sound effect information are collected, both of them can be used as the user reference sound effect information. Finally, the user reference sound effect information and each sound source sound effect information are fused to generate the scene sound effect information corresponding to the target virtual scenario.

[0095] In practical applications, according to the feedback instruction of the user (for example, the user points out that the sound effect transformation is too abrupt), the transition discomfort segment can be marked, and thus the user reference sound effect information can be adjusted.

[0096] When synthesizing the scene sound effect information, each sound source sound effect information and the user reference sound effect information can be input into the game engine, and the audio stream transmission can be realized through the game engine. An environmental acoustic model can also be established to adjust the environmental reference sound effect according to the user's environmental information. When a sound effect anomaly is detected, the sound effect can be switched to the default sound effect template, so as to achieve the fusing of abnormal sound effects and avoid the bad user experience brought by abrupt sound effects.

[0097] Through the method provided by the embodiments of the present application, the sound effect is synthesized in real time and dynamically through multiple sound source parameter information in the target virtual scenario, getting rid of the dependence on the fixed sound effect library. At the same time, multiple sound source parameter information generates multiple sound source sound effect information, and the fusion between multiple sound source sound effect information enriches the type of sound effect information and improves the user experience.

[0098] Furthermore, the multiple sound source parameter information includes environmental parameter information, character behavior information, and also includes user configuration information, supporting the multi-level configuration of the user for sound effect synthesis and improving the personalized user experience. This method also does not require the transformation of the underlying architecture. In the method provided by the present application, a hierarchical synthesis method based on the acoustic propagation model is also proposed. The environmental layer, the interaction layer, and the user configuration layer are combined to synthesize the final scene sound effect information.

[0099] In addition, this method also provides the feedback information of the user to adjust the sound effect synthesis rule, and adjusts the rule information through the ways of display feedback and implicit feedback.

[0100] The following combination of attachments Figure 2, taking the application of the virtual scene sound effect processing method provided by this application in the game scene as an example, the virtual scene sound effect processing method will be further described. Among them, Figure 2 Fig. Figure 2 shows a processing flow chart of a virtual scene sound effect processing method applied to a game scene provided by an embodiment of this application, which specifically includes the following steps: Step 202: Collect the environmental parameters, character behaviors, and user configurations of the game scene.

[0101] Among them, the environmental parameters of the game scene: the terrain is mountainous, the weather is heavy rain, and the time is night.

[0102] Character behavior: The moving speed is 5 m / s, in a running state, and the attack frequency is 4 times per second.

[0103] User configuration: Low-frequency gain +15%.

[0104] Step 204: Add terrain rules, weather rules, user configuration loading rules, and dynamic adjustment rules according to the terrain rules, and generate environmental sound effects, character sound effects, and configuration sound effects according to each rule.

[0105] Among them, the terrain is mountainous, and the terrain rule is to add a rock sliding sound effect (the volume changes with the mountain slope).

[0106] The weather is heavy rain, and the heavy rain rule is to superimpose the sound of rain (basic layer) + the sound of wind (high-pass filter, cut-off frequency 1 kHz).

[0107] The user configuration loading rule is to increase the volume of all low-frequency sound effects by 15%, and trigger high-frequency noise reduction when the attack frequency is greater than 3 times per second.

[0108] The dynamic adjustment rule is that the running speed of the character triggers the acceleration of the footstep rhythm (the tone is increased by 15%).

[0109] Step 206: According to the weighted scoring model, determine that the environmental weight is 0.5, the behavior weight is 0.3, the configuration weight is 0.2, and calculate the weight sum.

[0110] Specifically, the environmental parameters include a mountain, the environmental score is 1; the attack frequency of the character is greater than 3 times per second, the character score is 1; the user sets the low-frequency gain, and the configuration score is 1. Multiply each score by the corresponding weight and then add them up to obtain a weight sum of 1.

[0111] Step 208: If the weight sum is greater than the preset weight threshold, then fuse the environmental sound effect, character sound effect, and configuration sound effect and each weight to obtain the scene sound effect information.

[0112] Specifically, fuse the environmental sound effect according to the environmental weight, the character sound effect according to the behavior weight, and the configuration sound effect according to the configuration weight to generate the scene sound effect information.

[0113] The method provided in this embodiment synthesizes sound effects in real time based on environmental parameters and character behaviors, getting rid of the dependence on a fixed sound effect library and enriching the ways of generating sound effects. In addition, it combines user configuration to enhance the user's personalized configuration experience.

[0114] Figure 3 The flowchart of the sound effect processing method for a virtual scene provided by another embodiment of the present application is shown. As Figure 3 shown, the method includes the following steps: Step 302: Obtain user feature information and at least one sound source parameter information for the target virtual scene.

[0115] Step 304: Input the user feature information and each sound source parameter information into the sound effect synthesis model to obtain the scene sound effect information corresponding to the target virtual scene output by the sound effect synthesis model. Among them, the scene sound effect information is generated according to the user reference sound effect information and at least one sound source sound effect information. The user reference sound effect information is generated according to the user feature information, and at least one sound source sound effect information is generated according to at least one sound source parameter information.

[0116] In this embodiment, user feature information and at least one sound source parameter information for the target virtual scene are obtained, and the user feature information and each sound source parameter information are input into the sound effect synthesis model. The sound effect synthesis model is trained to output the scene sound effect information of the target virtual scene according to the user feature information and each sound source parameter information.

[0117] In practical applications, in order to make the scene sound effect information more suitable for the target virtual scene, the scene attribute information of the target virtual scene can also be obtained, and the scene attribute information is used as a prompt word to be input into the sound effect synthesis model together with the user reference sound effect information and each sound source sound effect information for processing, so that the scene sound effect information output by the sound effect synthesis model is more realistic.

[0118] The sound effect synthesis model can be a large language model that is fine-tuned in the sound effect synthesis scenario based on the large language model. Using deep learning technology, it adaptively fuses user information and the sound source parameter information of the target virtual scene to automatically generate high-fidelity, spatial and personalized scene sound effect information, significantly improving the efficiency of sound effect synthesis.

[0119] In a specific embodiment provided by the present application, the sound source sound effect information is generated in the following manner: Input each sound source parameter information into the sound effect synthesis model, so that the sound effect synthesis model determines the sound source sound effect rules corresponding to each sound source parameter information in the preset rule mapping library, and generates at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information.

[0120] In this embodiment, a preset rule mapping library that cooperates with the sound effect synthesis model can be preset. The sound source sound effect rules corresponding to the sound source parameter information are stored in the preset rule mapping library. During the process of the sound effect synthesis model processing the sound source parameter information, the sound source sound effect rules corresponding to the sound source parameter information are determined from the preset rule mapping library according to the sound source parameter information, and the sound source sound effect information is generated based on the sound source sound effect rules and the sound source parameter information. In practical applications, one sound source parameter information corresponds to one sound source sound effect rule, and the corresponding sound source parameter information and sound source sound effect rule generate the sound source sound effect information.

[0121] In another specific embodiment provided by this application, user characteristic information is obtained, including: Obtaining user environment information and / or user emotion information; Correspondingly, generating user reference sound effect information according to the user characteristic information includes: Inputting the user environment information and / or the user emotion information into a user parameter sound effect model to obtain the user reference sound effect information output by the user parameter sound effect model.

[0122] In this embodiment, the user parameter sound effect model can also be adaptively trained for the user characteristic information. This model is used to automatically output the user reference sound effect information according to the user characteristic information (such as user environment information, user emotion information, etc.). By making full use of the understanding and analysis ability of the large language model, the generated user reference sound effect information is more in line with the user's usage scenario.

[0123] Through the method provided by the embodiments of this application, when generating the scene sound effect of the target virtual scene, the user reference sound effect information can be generated according to the user's user characteristic information, the sound source sound effect information can be synthesized in real time and dynamically according to multiple sound source parameter information in the target virtual scene, and then the final scene sound effect information is generated through the user reference sound effect information and the sound source sound effect information. That is, it refers to the user's user characteristic information and can get rid of the dependence on the fixed sound effect library. At the same time, multiple sound source parameter information generates multiple sound source sound effect information, and the fusion between multiple sound source sound effect information enriches the type of sound effect information and improves the user's usage experience.

[0124] Corresponding to the above method embodiment, this application also provides an embodiment of a sound effect processing device for a virtual scene. Figure 4 The structural schematic diagram of a sound effect processing device for a virtual scene provided by an embodiment of this application is shown. As Figure 4 shown, the device includes: An obtaining module 402, configured to obtain at least one sound source parameter information for a target virtual scene; A determination module 404, configured to determine a sound source sound effect rule corresponding to each sound source parameter information in a preset rule mapping library, and determine at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information; A synthesis module 406, configured to synthesize the scene sound effect information of the target virtual scene according to each sound source sound effect information.

[0125] Optionally, the apparatus further includes a feedback module, configured to: Receive sound effect feedback information for the scene sound effect information; Determine a sound source sound effect rule to be adjusted according to the sound effect feedback information, and update the sound source sound effect rule to be adjusted based on the sound effect feedback information.

[0126] Optionally, the sound effect feedback information includes user rating information and / or user operation information; The feedback module is further configured to: Determine a sound source sound effect rule to be adjusted and a rule adjustment strategy according to the user rating information, and update the sound source sound effect rule to be adjusted based on the rule adjustment strategy; When the user operation information meets the rule adjustment condition, determine a sound source sound effect rule to be adjusted and a rule adjustment strategy according to the user operation information, and update the sound source sound effect rule to be adjusted based on the rule adjustment strategy.

[0127] Optionally, the obtaining module 402 is further configured to: Obtain at least one of environment parameter information, character behavior information, and user configuration information for a target virtual scene.

[0128] Optionally, the obtaining module 402 is further configured to: Obtain the environment parameter information of the target virtual scene according to the virtual scene engine corresponding to the target virtual scene; Obtain the character behavior information of the target virtual scene according to the character controller corresponding to the target virtual scene; Provide a user configuration page, where the user configuration page includes sound effect configuration parameter information for the target virtual scene; In response to operation information for the user configuration interface, obtain user configuration information for the target virtual scene.

[0129] Optionally, the sound source parameter information includes at least one of environment parameter information, character behavior information, and user configuration information; The determination module 404 is further configured to: Analyze the environmental parameter information to obtain at least one environmental parameter value, and determine the environmental sound effect rules corresponding to the environmental parameter values in the preset rule mapping library according to each environmental parameter value; Analyze the character behavior information to obtain at least one character behavior value, and determine the character sound effect rules corresponding to the character behavior values in the preset rule mapping library according to each character behavior value; Analyze the user configuration information to obtain at least one user configuration value, and determine the user configuration rules corresponding to the user configuration values in the preset rule mapping library according to each user configuration value.

[0130] Optionally, the determining module 404 is further configured to: Determine the target sound source sound effect rule and the target sound source parameter information corresponding to the target sound source sound effect rule, where the target sound source sound effect rule is any one of the sound source sound effect rules; Extract the target sound source sound effect formula in the target sound source sound effect rule; Determine the target sound source sound effect information corresponding to the target sound source sound effect rule according to the target sound source sound effect formula and the target sound source parameter information.

[0131] Optionally, the synthesizing module 406 is further configured to: Determine the sound source parameter weights corresponding to the sound source sound effect information according to the sound source parameter weights corresponding to the sound source parameter information; Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information and the sound source parameter weights.

[0132] Optionally, the synthesizing module 406 is further configured to: Statistically calculate the sum of the weights of the sound source parameter weights corresponding to the sound source sound effect information; In the case where the sum of the weights is greater than the preset weight threshold, synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information and the sound source parameter weights; In the case where the sum of the weights is less than or equal to the preset weight threshold, synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information.

[0133] Optionally, the device further includes a user sound effect module, which is configured to: Obtain user characteristic information; Generate user reference sound effect information according to the user characteristic information; The synthesizing module 406 is further configured to: Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information and the user reference sound effect information.

[0134] Optionally, the user sound effect module is further configured to: Obtain user environment information and / or user emotion information; Correspondingly, optionally, the user sound effect module is further configured to: Generate environmental reference sound effect information according to the user environment information; Generate emotion reference sound effect information according to the user emotion information; Generate user reference sound effect information according to the environmental reference sound effect information and / or emotion reference sound effect information.

[0135] Optionally, the user environment information includes an environmental map and environmental status data; The user sound effect module is further configured to: Obtain the user behavior information of the user; Generate environmental sound effect parameters according to the user behavior information and the environmental status information; Generate environmental reference sound effect information according to the environmental map and the environmental sound effect parameters.

[0136] Optionally, the user sound effect module is further configured to: Collect multi-dimensional biometric information of the user, and generate user emotion information according to the multi-dimensional biometric information, where the user emotion information includes basic dimension information and composite dimension information; Correspondingly, the user sound effect module is further configured to: Determine emotion sound effect parameter information and emotion sound effect fusion mode information corresponding to the user emotion information in a preset emotion parameter database according to the user emotion information.

[0137] Through the device provided in the embodiments of the present application, the sound effect is synthesized in real time and dynamically through multiple sound source parameter information in the target virtual scene, getting rid of the dependence on a fixed sound effect library. At the same time, multiple sound source parameter information generates multiple sound source sound effect information, and the fusion between multiple sound source sound effect information enriches the type of sound effect information and improves the user experience.

[0138] Furthermore, the multiple sound source parameter information includes environmental parameter information, character behavior information, and also includes user configuration information, supporting multi-level configuration of sound effect synthesis by the user and improving the personalized user experience. This device also does not need to modify the underlying architecture. In the device provided in the present application, a hierarchical synthesis based on an acoustic propagation model is also proposed. The environmental layer, interaction layer, and user configuration layer are combined to synthesize the final scene sound effect information.

[0139] In addition, this device also provides feedback information of the user to adjust the sound effect synthesis rule, and adjusts the rule information through explicit feedback and implicit feedback.

[0140] The above is a schematic solution of a sound effect processing device for a virtual scene in this embodiment. It should be noted that the technical solution of the sound effect processing device for the virtual scene and the technical solution of the above-mentioned sound effect processing method for the virtual scene belong to the same concept. For the details not described in detail in the technical solution of the sound effect processing device for the virtual scene, reference can be made to the description of the technical solution of the sound effect processing method for the virtual scene above.

[0141] Figure 5 The structural block diagram of a computing device 500 provided according to an embodiment of the present application is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 through a bus 530, and the database 550 is used to store data.

[0142] The computing device 500 further includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include the Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 540 may include one or more of any type of wired or wireless network interfaces (for example, a network interface card (NIC)), such as an IEEE802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0143] In an embodiment of the present application, the above components of the computing device 500 and Figure 5 other components not shown in Figure 5 may also be connected to each other, for example, through a bus. It should be understood that

[0144] The computing device 500 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.) or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). The computing device 500 can also be a mobile or stationary server.

[0145] Wherein, the processor 520 is used to execute the following computer program / instructions, and when the computer program / instructions are executed by the processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0146] The above is a schematic solution of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the sound effect processing method for the virtual scene described above belong to the same concept. For the details not described in detail in the technical solution of the computing device, reference can be made to the description of the technical solution of the sound effect processing method for the virtual scene described above.

[0147] An embodiment of this application further provides a computer-readable storage medium, which stores computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0148] The above is a schematic solution of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the sound effect processing method for the virtual scene described above belong to the same concept. For the details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solution of the sound effect processing method for the virtual scene described above.

[0149] An embodiment of this application further provides a computer program product, including computer program / instructions, and when the computer program / instructions are executed by a processor, the steps of the sound effect processing method for the virtual scene described above are implemented.

[0150] The above is a schematic solution of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product and the technical solution of the sound effect processing method for the virtual scene described above belong to the same concept. For the details not described in detail in the technical solution of the computer program product, reference can be made to the description of the technical solution of the sound effect processing method for the virtual scene described above.

[0151] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0152] The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, removable hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0153] It should be noted that for the foregoing method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present application is not limited by the described action sequence, because according to the present application, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present application.

[0154] In the above embodiments, the descriptions of the various embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0155] The preferred embodiments of the present application disclosed above are only used to help explain the present application. The alternative embodiments do not elaborate on all the details and do not limit the invention to only the specific embodiments described. Obviously, many modifications and variations can be made according to the content of the present application. The present application selects and specifically describes these embodiments to better explain the principles and practical applications of the present application, so that those skilled in the art can well understand and utilize the present application. The present application is only limited by the claims and their full scope and equivalents.

Claims

1. A method for processing sound effects in a virtual scene, characterized in that including: Obtaining at least one sound source parameter information for a target virtual scene; Determining sound source sound effect rules corresponding to each sound source parameter information in a preset rule mapping library, and determining at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information; Synthesizing scene sound effect information of the target virtual scene according to each sound source sound effect information.

2. The method according to claim 1, wherein It also includes: Receiving sound effect feedback information for the scene sound effect information; Determining a sound source sound effect rule to be adjusted according to the sound effect feedback information, and updating the sound source sound effect rule to be adjusted based on the sound effect feedback information.

3. The method according to claim 2, wherein The sound effect feedback information includes user rating information and / or user operation information; Determining a sound source sound effect rule to be adjusted according to the sound effect feedback information, and updating the sound source sound effect rule to be adjusted based on the sound effect feedback information, including: Determining a sound source sound effect rule to be adjusted and a rule adjustment strategy according to the user rating information, and updating the sound source sound effect rule to be adjusted based on the rule adjustment strategy; When the user operation information meets the rule adjustment condition, determining a sound source sound effect rule to be adjusted and a rule adjustment strategy according to the user operation information, and updating the sound source sound effect rule to be adjusted based on the rule adjustment strategy.

4. The method according to claim 1, characterized in that, Obtaining at least one sound source parameter information for a target virtual scene, including: Obtaining at least one of environmental parameter information, character behavior information, and user configuration information for the target virtual scene.

5. The method according to claim 4, characterized in that Obtaining environmental parameter information for a target virtual scene, including: Obtaining the environmental parameter information of the target virtual scene according to the virtual scene engine corresponding to the target virtual scene; Obtaining character behavior information for a target virtual scene, including: Obtaining the character behavior information of the target virtual scene according to the character controller corresponding to the target virtual scene; Obtaining user configuration information for a target virtual scene, including: Providing a user configuration page, where the user configuration page includes sound effect configuration parameter information for the target virtual scene; Responding to operation information for the user configuration interface, and obtaining user configuration information for the target virtual scene.

6. The method according to claim 1, wherein The sound source parameter information includes at least one of environmental parameter information, character behavior information, and user configuration information; Determining sound source sound effect rules corresponding to each sound source parameter information in a preset rule mapping library, including: Analyzing the environmental parameter information to obtain at least one environmental parameter value, and determining environmental sound effect rules corresponding to each environmental parameter value in the preset rule mapping library according to each environmental parameter value; Analyzing the character behavior information to obtain at least one character behavior value, and determining character sound effect rules corresponding to each character behavior value in the preset rule mapping library according to each character behavior value; Analyzing the user configuration information to obtain at least one user configuration value, and determining user configuration rules corresponding to each user configuration value in the preset rule mapping library according to each user configuration value.

7. The method according to claim 1, wherein Determining at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information, including: Determine the target sound source sound effect rule and the target sound source parameter information corresponding to the target sound source sound effect rule, where the target sound source sound effect rule is any one of the sound source sound effect rules; Extract the target sound source sound effect formula in the target sound source sound effect rule; Determine the target sound source sound effect information corresponding to the target sound source sound effect rule according to the target sound source sound effect formula and the target sound source parameter information.

8. The method according to claim 1, wherein Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source, including: Determine the sound source parameter weight corresponding to each sound source sound effect information according to the sound source parameter weight corresponding to each sound source parameter information; Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source and the sound source parameter weight of each sound source.

9. The method according to claim 8, wherein Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source and the sound source parameter weight of each sound source, including: Statistical sum of weights of the sound source parameter weights corresponding to the sound source sound effect information of each sound source; In the case where the sum of weights is greater than the preset weight threshold, synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source and the sound source parameter weight of each sound source; In the case where the sum of weights is less than or equal to the preset weight threshold, synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source.

10. The method according to claim 1, wherein Also includes: Obtain user characteristic information; Generate user reference sound effect information according to the user characteristic information; Correspondingly, synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source, including: Synthesize the scene sound effect information of the target virtual scene according to the sound source sound effect information of each sound source and the user reference sound effect information.

11. The method according to claim 10, wherein Obtain user characteristic information, including: Obtain user environment information and / or user emotion information; Correspondingly, generate user reference sound effect information according to the user characteristic information, including: Generate environmental reference sound effect information according to the user environment information; Generate emotion reference sound effect information according to the user emotion information; Generate user reference sound effect information according to the environmental reference sound effect information and / or the emotion reference sound effect information.

12. The method according to claim 11, wherein The user environment information includes an environmental map and environmental status data; Generate environmental reference sound effect information according to the user environment information, including: Obtain the user behavior information of the user; Generate environmental sound effect parameters according to the user behavior information and the environmental status information; Generate environmental reference sound effect information according to the environmental map and the environmental sound effect parameters.

13. The method according to claim 11, characterized in that, Obtain user emotion information, including: Collect the multi-dimensional biometric information of the user, and generate user emotion information according to the multi-dimensional biometric information, where the user emotion information includes basic dimension information and composite dimension information; Correspondingly, generate emotion reference sound effect information according to the user emotion information, including: Determine the emotion sound effect parameter information and the emotion sound effect fusion mode information corresponding to the user emotion information in the preset emotion parameter database according to the user emotion information.

14. A method for processing sound effects in a virtual scene, characterized in that, Includes: Obtain user characteristic information and at least one sound source parameter information for the target virtual scene; Input the user characteristic information and each sound source parameter information into the sound effect synthesis model to obtain the scene sound effect information corresponding to the target virtual scene output by the sound effect synthesis model, wherein the scene sound effect information is generated according to the user reference sound effect information and at least one sound source sound effect information, the user reference sound effect information is generated according to the user characteristic information, and at least one sound source sound effect information is generated according to at least one sound source parameter information.

15. The method according to claim 14, wherein The sound source sound effect information is generated by the following method: Input each sound source parameter information into the sound effect synthesis model, so that the sound effect synthesis model determines the sound source sound effect rules corresponding to each sound source parameter information in the preset rule mapping library, and generates at least one sound source sound effect information according to each sound source sound effect rule and each sound source parameter information.

16. The method according to claim 14, characterized in that, Obtain user characteristic information, including: Obtain user environment information and / or user emotion information; Correspondingly, generating user reference sound effect information according to the user characteristic information includes: Input the user environment information and / or the user emotion information into the user parameter sound effect model to obtain the user reference sound effect information output by the user parameter sound effect model.

17. A computer-readable storage medium stores computer programs / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

18. A computer program product, comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 16 are implemented.

Images