Audio rendering method, storage medium and electronic device

Abstract

The embodiment of the invention provides an audio rendering method, a storage medium and an electronic device. The method comprises the following steps: identifying target audio data corresponding to a target sound effect scene from collected sample audio data; and determining a corresponding target sound mixing parameter according to the target sound effect scene and the target audio data, and performing audio rendering on the target audio data based on the target sound mixing parameter. Through the audio rendering method and device, the problem that an existing audio rendering method cannot be used for rendering specific scenes is solved, and the effect of improving the reality sense of the rendered audio is achieved.

Description

Technical Field

[0001] The embodiments of the present invention relate to the technical field of audio processing, and in particular, to an audio rendering method, a storage medium and an electronic device. Background Art

[0002] Currently, the recognition of specific game scenes basically uses images to match the game screen. Some solutions use audio features to identify game scenes. For example, DTS (Digital Theater Systems) frequency modulation is used to gain audio data. This is two-dimensional sound frequency modulation, which only increases a certain frequency band. The enhanced sound effect is general, and the rendering audio technology is not rendered for specific scenes. Summary of the invention

[0003] The embodiments of the present invention provide an audio rendering method, a storage medium and an electronic device, so as to at least solve the problem that the existing audio rendering method in the related art cannot render for a specific scene.

[0004] According to one embodiment of the present invention, there is provided an audio rendering method, comprising: identifying target audio data corresponding to a target sound effect scene from collected sample audio data; wherein the target audio data comprises audio data of multiple audio tracks; determining corresponding target mixing parameters according to the target sound effect scene and the target audio data, and performing audio rendering on the target audio data based on the target mixing parameters.

[0005] According to yet another embodiment of the present invention, a computer-readable storage medium is provided, in which a computer program is stored, wherein the computer program is configured to execute the steps of any one of the above method embodiments when run.

[0006] According to yet another embodiment of the present invention, there is provided an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0007] Through the above-mentioned embodiments of the present invention, the target audio data corresponding to the target sound effect scene can be identified from the collected sample audio data. When the target sound effect scene is a specific sound effect scene, the target audio data of the specific sound effect scene is identified, and then the audio data of the specific sound effect scene can be rendered. Therefore, the problem that the existing audio rendering method cannot render for a specific scene can be solved, and the effect of improving the realism of the rendered audio can be achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Figure 1It is a schematic diagram of a hardware structure block diagram of a mobile terminal for running an audio rendering method according to an embodiment of the present invention;

[0009] Figure 2 is a flowchart of an audio rendering method according to an embodiment of the present invention;

[0010] Figure 3 is a schematic diagram of a structural block diagram of an audio rendering device according to an embodiment of the present invention;

[0011] Figure 4 is a flowchart of a method for detecting a specific game scene according to an embodiment of the present invention;

[0012] Figure 5 is a schematic flow chart of a spatial rendering method according to an embodiment of the present invention;

[0013] Figure 6 is a schematic diagram of superimposing multi-track data according to an embodiment of the present invention;

[0014] Figure 7 is a schematic diagram of a synthesized waveform according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in combination with the embodiments.

[0016] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0017] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 Schematic diagram of a hardware structure block diagram of a mobile terminal running an audio rendering method according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0018] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the audio rendering method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0019] The transmission device 106 is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0020] In this embodiment, an audio rendering method running on the above mobile terminal is provided. Figure 2 is a flow chart of an audio rendering method according to an embodiment of the present invention. Figure 2 As shown, the process may include the following steps:

[0021] Step S202: identifying target audio data corresponding to a target sound effect scene from the collected sample audio data; wherein the target audio data may include audio data of one or more audio tracks.

[0022] In an exemplary embodiment, step S202 may include: identifying the current sound effect scene according to specific sound features corresponding to the game scene, and determining the identified sound effect scene as the target sound effect scene; classifying the sample audio data according to different sound effect scenes; and identifying the target audio data of the target sound effect scene based on the classified sample audio data.

[0023] In one embodiment, classifying sample audio data according to different sound effect scenes may include: performing audio data processing on the sample audio data to obtain a Mel-cepstral coefficient map; inputting the Mel-cepstral coefficient map into a deep convolutional network model to classify the sample audio data according to the different sound effect scenes; wherein the activation function used in the activation layer of the deep convolutional network model is a hyperbolic tangent function, and the hyperbolic tangent function includes a first adjustable parameter and a second adjustable parameter, and the first adjustable parameter and the second adjustable parameter are used to adjust the shape of the activation function.

[0024] The hyperbolic tangent function has higher flexibility. By adjusting the first adjustable parameter and the second adjustable parameter, the shape of the activation function can be adjusted to adapt to different types of problems. In addition, the hyperbolic tangent function has more analytical derivatives, which can be more convenient to derive and optimize. When the second adjustable parameter is larger, the gradient of the hyperbolic tangent function near Y=0 is larger, which is beneficial to avoid the gradient vanishing problem in the neural network.

[0025] In one embodiment, performing audio data processing on the sample audio data to obtain a Mel-cepstral coefficient map may include: preprocessing the sample audio data according to a preset preprocessing order, wherein the preprocessing includes at least one of the following: pre-emphasis, framing, windowing, and Fourier transform, to obtain a frequency spectrum corresponding to the sample audio data; and converting the frequency spectrum into the Mel-cepstral coefficient map through a Mel filter group.

[0026] In one embodiment, the Mel-cepstral coefficient map is input into a deep convolutional network model to classify the sample audio data according to different sound effect scenes, including: adjusting the shape of the activation function so that the deep convolutional network model can recognize the audio data of different sound effect scenes to classify the sample audio data according to the different sound effect scenes.

[0027] By using the Mel-frequency cepstral coefficient map as the input of the deep convolutional network, the game-specific scenes are automatically marked through the deep convolutional network, and the predictive ability of the nonlinear activation function is utilized to obtain more accurate classification results.

[0028] Step S204: determining corresponding target mixing parameters according to the target sound effect scene and the target audio data, and performing audio rendering on the target audio data based on the target mixing parameters.

[0029] After the audio data is rendered using the target mixing parameters, the audio output by the terminal device can have a spatial sound effect, thereby improving the user's gaming experience.

[0030] In one embodiment, the corresponding target mixing parameters are determined according to the target sound effect scene and the target audio data, including: the target sound effect scene corresponds to a standard audio parameter, the standard audio parameter is compared with the target audio data, and the target mixing parameters are determined, and the target mixing parameters can make the target audio data more stereoscopic.

[0031] In an exemplary embodiment, in step S204, performing stereo audio rendering on the target audio data based on the target mixing parameters may include: linearly superimposing the audio data of the multiple audio tracks in multiple audio channels respectively, and adjusting the sound wave amplitude of the superimposed audio data in each of the audio channels according to the target mixing parameters to obtain rendered synthetic audio track data, wherein the target mixing parameters include: the sound wave amplitude of each audio channel.

[0032] In one embodiment, before linearly superimposing the audio data of the multiple audio tracks in multiple audio channels respectively, the method further includes: providing the user with angle adjustment information of the terminal device and / or distance adjustment information of the terminal device according to the target sound effect scene, so as to prompt the user to adjust the terminal device to a position that meets a preset position condition.

[0033] After step S204 of this embodiment, the method may further include: adjusting the volume of the left channel and the right channel of the terminal device according to the current position of the terminal device.

[0034] Through the above steps, the target audio data corresponding to the target sound effect scene can be identified from the collected sample audio data. When the target sound effect scene is a specific sound effect scene, the audio data of the specific sound effect scene is identified, and then the audio rendering can be performed on the target audio data of the specific sound effect scene. Therefore, the problem that the existing audio rendering method cannot render for a specific scene can be solved, and the effect of improving the realism of the rendered audio can be achieved.

[0035] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0036] In this embodiment, an audio rendering device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware of a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0037] Figure 3 is a schematic diagram of a structural block diagram of an audio rendering device according to an embodiment of the present invention, the device is located on a user's terminal device, such as Figure 2 As shown, the device may include: a recognition module 10 and a rendering module 20.

[0038] The recognition module 10 is used to recognize audio data corresponding to the target sound effect scene from the collected sample audio data; wherein the audio data includes audio data of multiple tracks;

[0039] The rendering module 20 is used to determine corresponding target mixing parameters according to the target sound effect scene and the audio data, and perform audio rendering on the audio data based on the target mixing parameters.

[0040] It should be noted that the above modules can be implemented by software or hardware. For the latter, it can be implemented in the following ways, but not limited to: the above modules are all located in the same processor; or the above modules are located in different processors in any combination.

[0041] To facilitate the understanding of the technical solution provided by the present invention, a detailed description will be given below in conjunction with embodiments of specific scenarios.

[0042] The technology of the present invention provides an audio feature extraction and spatial audio rendering method, by which the user's gaming experience can be improved. For example, in top games that have already appeared, such as Peace Elite, Honor of Kings, and Genshin Impact, the method can help users optimize the audio effects of the enemy's skill release before a gunfight or before the enemy releases a skill, render and amplify the enemy's footsteps, and render and spatialize these sounds to transmit them to game users, allowing users to seize the initiative before the game battle, have a keener perception, and bring a more realistic and immersive experience; at the same time, it is also suitable for applications that play sound effects for non-game types.

[0043] The audio feature extraction and spatial audio rendering method of the embodiment of the present invention mainly involves audio feature processing technology and spatial audio technology. The following briefly describes the two technologies.

[0044] 1. Audio feature processing

[0045] Most audio features originally originated from speech recognition and can simplify the original waveform sampling signals so that they can be used in other models, making it easier for algorithms to understand the semantic information contained in the audio.

[0046] Audio data processing algorithms typically involve analyzing signals, extracting their properties, predicting their behavior, identifying whether there is a certain pattern in the signal or how a specific signal is related to another similar signal, where audio data includes music, speech, and environmental sounds.

[0047] Many existing problems can be solved by combining modern machine learning algorithms with audio data processing techniques. The performance of a machine learning algorithm depends on the features used to complete the training and testing. Therefore, feature extraction is one of the most important parts of the machine learning process.

[0048] 2. Spatial Audio

[0049] From mono to dual-channel, then to multi-channel and surround sound, the expressiveness of digital audio is becoming more and more vivid. Spatial audio, also known as three-dimensional sound or 3D audio, is not simply to make the sound more three-dimensional by adding more channels, but the spatialization process of audio synchronized with the spatialization of video. It can be applied to music, movies, games and other scenes, as well as online office, XR conference, remote assistance, 5G and other immersive interactive scenes.

[0050] Existing audio rendering technology does not render for specific scenes. The audio feature extraction and spatial audio rendering method in the embodiment of the present invention renders and spatializes game-specific scenes and delivers them to game users. This method combines audio feature extraction technology and mixing simulation spatial audio rendering technology, and uses a mixing simulation space system to achieve the rendering of spatial audio in specific game scenes, while reconstructing the reverberation and noise environment to achieve the spatial presentation of audio. After audio gain, the effect is more three-dimensional and the experience is better. Not only can three-dimensional sound effects be achieved, but also an immersive audio environment can be provided.

[0051] The audio feature extraction and spatial audio rendering method in this embodiment may include: obtaining audio data through the underlying audio framework, classifying the sound by extracting audio features, obtaining audio clips of a specific game scene, performing secondary feature extraction through deep convolution, and then reporting the extracted audio features to the game framework, which matches the game scene in the feature library, and after the user enters the game, after identifying the sound features of the specific game scene, obtains the audio clip, simulates the spatial audio effect through mixing, and then linearly modulates the audio in the specific game scene to render the audio effect. It is required that the game application or non-game application has sound effect playback.

[0052] Among them, feature extraction is to obtain a set of feature parameters that characterize the change of speech signals over time by analyzing speech signals. The feature parameters extracted by different feature extraction methods directly affect the classification effect of game scenes and are the key steps to extract specific game scenes. Commonly used features include: time domain features, frequency domain features, cepstrum features, etc., but these extraction methods all have certain problems. MFCC (Mel Frequency Cepstrum Coefficient) will have a significant drop in performance when the signal-to-noise ratio is relatively low, and the anti-noise performance is poor and the robustness is poor. Considering the success of deep convolutional networks in speech recognition, image and other fields, in this embodiment, for the extraction of specific game scene audio, after the spectrum extracted by MFCC is input into the deep convolutional network, features will be further extracted, making the classification effect more obvious.

[0053] Different from the existing audio rendering technology that sets audio rendering parameters through acoustic environment information, this embodiment uses audio feature preprocessing, reads scene information, and then mixes to simulate spatial audio effects to extract audio elements required by the renderer, and then decodes them to the playback device according to the user scenario. At the same time, different frequency response curves and gains are set for different devices. This solution combines audio features with spatial audio rendering to interact with users, making users' perception more acute and bringing a more realistic and immersive experience. It not only enhances the user experience in terms of hearing, but also improves the user's gaming experience.

[0054] Figure 4 is a flowchart of a method for detecting a specific game scene according to an embodiment of the present invention. The method can be based on Mel-frequency cepstral coefficient features and a deep convolutional network, such as Figure 4 As shown, the method may include the following steps:

[0055] Step S401, obtaining audio data in the current game scene through the terminal device.

[0056] Step S402: pre-emphasize and perform overlapping frame windowing on the collected audio data.

[0057] In one embodiment, step S402 may also be: high-pass filtering the audio data and dividing the audio data into frames in an overlapping manner.

[0058] Step S403, performing Fourier transform on each short-time analysis window to obtain a corresponding frequency spectrum.

[0059] Step S404, processing the frequency spectrum obtained in step S403 by a MEL filter group to obtain a Mel cepstrum.

[0060] Step S405: input the Mel-frequency cepstral coefficient map into the trained deep convolutional network model.

[0061] Inputting the Mel-frequency cepstral coefficient graph into a trained deep convolutional network model, including:

[0062] Step S4051, the input layer represents the frequency spectrum of the input deep convolutional network in matrix form.

[0063] In one embodiment, the size of the matrix is ​​a matrix of M(3)×N(3), where M is the number of points on the frequency axis, and N is the number of points on the time axis, and M and N are integers greater than zero.

[0064] Step S4052: The convolution layer performs a convolution operation on the output K feature maps.

[0065] Specifically, the convolution operation includes: sliding a small-sized convolution kernel on the spectrum graph to capture local features; for example: there are K convolution kernels, the size of each convolution kernel is F x F, the step size of the convolution operation is S, and finally K feature maps are output in the convolution layer, where F, S, and K are integers greater than zero.

[0066] In one embodiment, given an input spectrogram X, a convolution kernel W and a bias term b, the convolution operation can be expressed as: Y=X*W+b, where * represents the convolution operation.

[0067] Step S4053: The activation layer applies a nonlinear activation function to the K feature maps output by the convolutional layer.

[0068] In one embodiment, the non-linear activation function is Leaky ReLU.

[0069] In one embodiment, the nonlinear activation function is HTO-ReLU, also known as hyperbolic tangent function, which is expressed as follows:

[0070] f(Y)'=A*tanh(B*Y)

[0071] Among them, A and B are adjustable parameters that can be optimized according to actual problems. In the current model, when A=1 and B=1, the HTO-ReLU function degenerates into the commonly used hyperbolic tangent function.

[0072] The commonly used ReLU function is ReLU(x)=max(0,x)

[0073] In this embodiment, the improved HTO-ReLU function is more flexible. By adjusting parameters A and B (equivalent to the first adjustable parameter and the second adjustable parameter in the above embodiment), the shape of the activation function can be adjusted to adapt to different types of problems; compared with the commonly used activation functions (RelU and Sigmoid), the HTO-ReLU function has more analytical derivatives, and it is more convenient to obtain derivatives and optimize; when parameter B is larger, the gradient of the HTO-ReLU function near Y=0 is larger, which can avoid the gradient vanishing problem in the neural network.

[0074] Step S4054: The pooling layer performs a pooling operation on the output result of the activation layer.

[0075] For example: by using maximum pooling or average pooling to reduce the size of the feature map while retaining key features, the size of the pooling kernel is P x P, the step size of the pooling operation is D, and P and D are integers greater than zero.

[0076] The pooling operation is expressed as: Z = pool (Y'), where pool represents average pooling.

[0077] Step S4055: The fully connected layer flattens the output of the pooling layer into a one-dimensional vector and performs the classification task through the fully connected layer, wherein the number of nodes output by the fully connected layer is the number of categories.

[0078] In one embodiment, the output of the pooling layer is flattened into a one-dimensional vector V and a linear combination is calculated to obtain:

[0079] U = V*W' + b', where W' and b' are the weight and bias of the fully connected layer, respectively.

[0080] Step S4056: The output layer converts the output of the fully connected layer into a probability distribution through the Softmax function for final classification.

[0081] In one embodiment, a Softmax function is applied at the output layer to convert U into a probability distribution P:

[0082] P(i)=exp(U(i)) / sum(exp(U(j))).

[0083] Step S406, recording the detected audio categories, and classifying and outputting the specific game scenes.

[0084] Through the above steps, game scene-specific audio features can be extracted from the spectrogram, and effective game-specific sound scene classification tasks can be performed. Deep convolution is used to automatically label game-specific scenes, and the cepstrum is used as the input of the deep convolutional network. The nonlinear activation function is used to make the network predictive, and the classification accuracy is greatly improved compared to MFCC.

[0085] Existing spatial audio technology mainly uses hardware equipment to replicate the sound field from the acquisition level, and then uses playback equipment to restore the sound field. This requires a full set of equipment to arrange the spatial environment, and most users cannot afford expensive equipment; some use headphones to simulate and restore the sense of space at the playback end. These require users to have additional equipment in addition to the terminal device to experience the effect of spatial audio. The spatial audio rendering in the embodiment of the present invention can only rely on software algorithms to mix and simulate and restore the sense of space on the terminal device. Among them, the principle of mixing and simulating the restoration of the sense of space is:

[0086] When the sound is transmitted to different speakers, the sound becomes more three-dimensional and spatial. Based on this principle, the sound data of multiple channels are linearly superimposed, and each channel of audio data is weighted according to its own proportion. After synthesis, it is output according to the weight, and then the volume and sound wave amplitude are adjusted to achieve the superposition of sound waves in the air, which simulates the effect of spatial sense when heard by human ears.

[0087] Figure 5 is a schematic diagram of a flow chart of a spatial rendering method according to an embodiment of the present invention. Figure 5 As shown, the method may include:

[0088] Step S501, based on the identification of a specific game scene, prompting the user to set the angle and distance of the sound source to obtain the location information of the sound source;

[0089] Step S502, based on the extracted audio data of the specific game scene, obtain sampled audio data, and linearly superimpose the values ​​of each channel of multiple audio track data.

[0090] The value generated by linearly superposing the values ​​of each channel is used as the synthesized value. Figure 6 is a schematic diagram of superimposing multi-track data according to an embodiment of the present invention, Figure 7 Schematic diagram of a synthetic waveform according to an embodiment of the present invention. Figure 6 As shown, it is the superposition of Track A, Track B and Track C. A.1 represents the value of channel 1 of Track A, AC02; B.1 represents the value of channel 1 of Track B, 43AD; C1 represents the value of channel 1 of Track C, 5544. The result is JKFK. Then it is arranged in the low-order first order. In the composite track, it is JKFK. It is represented in hexadecimal here.

[0091] Calculation formula: average value = (X1+X2+X3+....+Xn) / n.

[0092] Step S503, adjusting the sound wave amplitude of each audio channel through DTS to obtain adjusted synthesized audio track data.

[0093] Step S504, adjusting the volume of the left and right channels of the terminal according to the location information of the sound source, and obtaining an audio output with a simulated spatial effect.

[0094] Step S505: outputting the audio after simulating the spatial effect.

[0095] Through the above steps, the sound data of multiple channels are linearly superimposed, and each channel of audio data is weighted according to its own proportion. After synthesis, it is output according to the weight, and then the volume and the amplitude of the sound wave are adjusted to achieve the superposition of sound waves in the air, and the human ear simulates the effect of spatial sense when hearing it. The whole process is realized by software, without the need for additional equipment, and the implementation method is simple.

[0096] The embodiment of the present invention also provides a method for reconstructing a confusing and noisy environment to enhance the sense of space:

[0097] Step S1, based on the speaker array of the terminal device, a circular environment is set to collect sound field information;

[0098] Step S2, transforming and processing the sound field information accordingly through spherical harmonic decomposition;

[0099] Step S3, solving the speaker array signal and reconstructing the real sound field.

[0100] The present invention classifies sounds through audio feature extraction to obtain audio clips of specific game scenes. Based on the spatial audio rendering algorithm of neural network, linear frequency modulation enhances the audio effect in specific game scenes, provides an immersive audio environment, enhances the gameplay, and user experience.

[0101] The above-mentioned embodiments of the present invention can be applied to scenarios such as music, movies, and games, and can also be applied to immersive interactive scenarios such as online office, XR conferencing, remote assistance, and 5G communication.

[0102] An embodiment of the present invention further provides a computer-readable storage medium, in which a computer program is stored, wherein the computer program is configured to execute the steps of any of the above method embodiments when running.

[0103] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk or an optical disk.

[0104] An embodiment of the present invention further provides an electronic device, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0105] In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0106] For specific examples in this embodiment, reference may be made to the examples described in the above embodiments and exemplary implementation modes, and this embodiment will not be described in detail herein.

[0107] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0108] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. An audio rendering method, characterized in that: Applied to the terminal side, including: Identifying target audio data corresponding to a target sound effect scene from the collected sample audio data; The target audio mixing parameters corresponding to the target audio scene and the target audio data are determined, and audio rendering is performed on the target audio data based on the target audio mixing parameters.

2. The method according to claim 1, characterized in that From the collected sample audio data, identify the target audio data of the target sound effect scene, including: Identify the current sound effect scene according to the specific sound features corresponding to the game scene, and determine the identified sound effect scene as the target sound effect scene; Classifying the sample audio data according to different sound effect scenes; According to the classified sample audio data, target audio data of the target sound effect scene is identified.

3. The method according to claim 2, characterized in that Classify sample audio data according to different sound effect scenarios, including: Performing audio data processing on the sample audio data to obtain a Mel-frequency cepstral coefficient graph; The Mel-frequency cepstral coefficient map is input into a deep convolutional network model to classify the sample audio data according to different sound effect scenes; wherein the activation function used in the activation layer of the deep convolutional network model is a hyperbolic tangent function, and the hyperbolic tangent function includes a first adjustable parameter and a second adjustable parameter, and the first adjustable parameter and the second adjustable parameter are used to adjust the shape of the activation function.

4. The method according to claim 3, characterized in that Performing audio data processing on the sample audio data to obtain a Mel-frequency cepstral coefficient graph includes: Perform at least one of the following processing on the sample audio data: pre-emphasis, framing, windowing, Fourier transform, to obtain a frequency spectrum corresponding to the sample audio data; The spectrum is converted into the Mel cepstral coefficient map through a Mel filter bank.

5. The method according to claim 3, characterized in that: Inputting the Mel-frequency cepstral coefficient graph into a deep convolutional network model to classify the sample audio data according to different sound effect scenes, including: By adjusting the shape of the activation function, the deep convolutional network model can recognize audio data of different sound effect scenes, so as to classify the sample audio data according to the different sound effect scenes.

6. The method according to claim 1, characterized in that The target audio data includes audio data of multiple audio tracks, and audio rendering is performed on the target audio data based on the target mixing parameter, including: The audio data of the multiple audio tracks are linearly superimposed in multiple audio channels respectively, and the sound wave amplitude of the superimposed audio data in each audio channel is adjusted according to the target mixing parameters to obtain rendered synthetic audio track data, wherein the target mixing parameters include: the sound wave amplitude of each audio channel.

7. The method according to claim 6, characterized in that Before linearly superimposing the audio data of the multiple audio tracks in the multiple audio channels respectively, the method further includes: According to the target sound effect scene, the angle adjustment information of the terminal device and / or the distance adjustment information of the terminal device is provided to the user, so as to prompt the user to adjust the terminal device to a position that meets the preset position condition.

8. The method according to claim 7, characterized in that After performing audio rendering on the audio data based on the target mixing parameters, the method further includes: According to the current position of the terminal device, the volume of the left channel and the right channel of the terminal device are adjusted.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program implements the steps of the method described in any one of claims 1 to 8 when executed by a processor.

10. An electronic device comprising one or more memories, one or more processors, and a computer program stored in the memories and executable on the processors, characterized in that: When the processor executes the computer program, the steps of the method described in any one of claims 1 to 8 are implemented.

Images