Equipment sound effect processing method and device, electronic equipment and storage medium

Abstract

The invention relates to an equipment sound effect processing method and device, electronic equipment and a storage medium, and the method comprises the steps: determining a first energy value of a first sound channel input signal and a second energy value of a second sound channel input signal in response to the received first sound channel input signal and second sound channel input signal, according to the first energy value and the second energy value, the energy relative ratio of the first sound channel signal to the second sound channel signal is determined, according to the target energy value and the energy relative ratio, the gain parameter of the target sound channel signal is determined, the target sound channel signal is the sound channel signal with the larger energy value in the first sound channel signal and the second sound channel signal, and the gain parameter of the target sound channel signal is determined. The target energy value is the energy value of the target sound channel signal, and performing equalization filtering on the target sound channel signal based on the gain parameter to output the target audio signal. Therefore, energy value calculation and ratio calculation of the sound channel input signal are combined and then mapped to gain adjustment of the sound channel, and more accurate and immersive auditory experience is provided for a user.

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio processing, and particularly to a method and apparatus for processing device sound effects, an electronic device, and a storage medium. Background Art

[0002] Spatial sound effects play a crucial role in enhancing the user's auditory experience. Through sound effect simulation, it can not only provide information about the environment and the state of objects, but also help the user maintain the perception of the real environment and guide the user to complete corresponding operation tasks. Such sound effects enable the user to have a more realistic usage experience in terms of hearing by enhancing the interactive perception between the user and the virtual environment. Summary of the Invention

[0003] To overcome the problems existing in the related art, the present disclosure provides a method and apparatus for processing device sound effects, an electronic device, and a storage medium.

[0004] According to a first aspect of an embodiment of the present disclosure, a method for processing device sound effects is provided. The method includes:

[0005] In response to receiving a first-channel input signal and a second-channel input signal, determining a first energy value of the first-channel input signal and a second energy value of the second-channel input signal;

[0006] According to the first energy value and the second energy value, determining an energy relative ratio of the first-channel signal and the second-channel signal;

[0007] According to a target energy value and the energy relative ratio, determining a gain parameter of a target-channel signal, where the target-channel signal is the channel signal with a larger energy value among the first-channel signal and the second-channel signal, and the target energy value is the energy value of the target-channel signal;

[0008] Performing equalization filtering on the target-channel signal based on the gain parameter to output a target audio signal.

[0009] Optionally, the determining a gain parameter of a target-channel signal according to a target energy value and the energy relative ratio includes:

[0010] If the energy relative ratio is within a preset interval range, determining a first gain upper limit of the target-channel signal according to the energy relative ratio, and determining a second gain upper limit of the target-channel signal according to the target energy value;

[0011] Determining the gain parameter according to the first gain upper limit and the second gain upper limit;

[0012] If the relative energy ratio is not within the preset range, determine the gain parameter as the preset gain parameter.

[0013] Optionally, determining the first gain upper limit of the target channel signal according to the relative energy ratio includes:

[0014] Obtain the first maximum gain parameter of the target channel signal;

[0015] Determine the first gain upper limit according to the first maximum gain parameter and the relative energy ratio.

[0016] Optionally, determining the first gain upper limit according to the first maximum gain parameter and the relative energy ratio includes:

[0017] Determine the first gain upper limit through the following formula;

[0018]

[0019] Wherein, the gain1 is the first gain upper limit, the r is the relative energy ratio, and the maxGain1 is the first maximum gain parameter.

[0020] Optionally, determining the second gain upper limit of the target channel signal according to the target energy value includes:

[0021] Obtain the second maximum gain parameter of the target channel signal;

[0022] Determine the second gain upper limit according to the second maximum gain parameter and the target energy value.

[0023] Optionally, determining the first energy value of the first channel input signal and determining the second energy value of the second channel input signal includes:

[0024] Obtain the first historical energy value of the first channel input signal in the previous frame and the first forgetting factor of the channel input signal;

[0025] Determine the first initial energy value of the first channel input signal;

[0026] Perform autoregressive smoothing filtering on the first initial energy value according to the first historical energy value and the first forgetting factor to generate the first energy value;

[0027] Obtain the second historical energy value of the second channel input signal in the previous frame;

[0028] Determine the second initial energy value of the second channel input signal;

[0029] Perform autoregressive smoothing filtering on the second initial energy value according to the second historical energy value and the first forgetting factor to generate the second energy value.

[0030] Optionally, determining the relative energy ratio of the first channel input signal and the second channel input signal according to the first energy value and the second energy value includes:

[0031] Obtain the historical relative energy ratio of the previous frame of audio signal and the second forgetting factor of the target channel signal;

[0032] Determine an initial relative energy ratio according to the first energy value and the second energy value;

[0033] Perform autoregressive smoothing filtering on the initial relative energy ratio according to the historical relative energy ratio and the second forgetting factor to generate the relative energy ratio.

[0034] Optionally, the method further includes:

[0035] Obtain a first initial channel input signal and a second initial channel input signal;

[0036] Filter the first initial channel input signal and the second initial channel input signal according to a preset filtering frequency band to generate the first channel input signal and the second channel input signal.

[0037] Optionally, the preset filtering frequency band is 0 - 1000 Hz or 0 - 2000 Hz.

[0038] Optionally, determining the first energy value of the first channel input signal and determining the second energy value of the second channel input signal includes:

[0039] Obtain the device sound effect mode;

[0040] When it is determined that the device sound effect mode is the game scene mode, determine the first energy value and the second energy value.

[0041] Optionally, the first channel input signal is the left channel input signal, the second channel input signal is the right channel input signal, and / or,

[0042] The first energy value is the first short-time energy value, the second energy value is the second short-time energy value, and the target energy value is the target short-time energy value.

[0043] According to the second aspect of the embodiments of the present disclosure, there is provided a device sound effect processing apparatus, the apparatus includes:

[0044] A first determination module, configured to determine a first energy value of the first channel input signal and determine a second energy value of the second channel input signal in response to receiving the first channel input signal and the second channel input signal;

[0045] A second determination module, configured to determine an energy relative ratio of the first channel signal and the second channel signal according to the first energy value and the second energy value;

[0046] A third determination module, configured to determine a gain parameter of a target channel signal according to a target energy value and the energy relative ratio, where the target channel signal is the channel signal with a larger energy value among the first channel signal and the second channel signal, and the target energy value is the energy value of the target channel signal;

[0047] An execution module, configured to perform equalization filtering on the target channel signal based on the gain parameter to output a target audio signal.

[0048] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, characterized by including:

[0049] A memory, on which a computer program is stored;

[0050] A processor, configured to execute the computer program in the memory to implement the steps of the device sound effect processing method according to any one of the first aspects of the present disclosure.

[0051] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the device sound effect processing method provided in the first aspect of the present disclosure are implemented.

[0052] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:

[0053] In the above manner, in response to receiving the first channel input signal and the second channel input signal, the first energy value of the first channel input signal and the second energy value of the second channel input signal are determined. According to the first energy value and the second energy value, the energy relative ratio of the first channel signal and the second channel signal is determined. According to the target energy value and the energy relative ratio, the gain parameter of the target channel signal is determined, where the target channel signal is the channel signal with a larger energy value among the first channel signal and the second channel signal, and the target energy value is the energy value of the target channel signal. Equalization filtering is performed on the target channel signal based on the gain parameter to output a target audio signal. Thus, the calculation of the energy value of the channel input signal and the ratio calculation are combined, and then mapped to the gain adjustment of the channel, enhancing the direction positioning ability of the channel input signal in the terminal and providing a more accurate and immersive auditory experience for the user.

[0054] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and should not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The drawings herein are incorporated into and constitute a part of this specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure.

[0056] Figure 1 is a flowchart of a method for processing device sound effects according to an exemplary embodiment.

[0057] Figure 2 is a flowchart of a method for determining a gain parameter according to an exemplary embodiment.

[0058] Figure 3 is a schematic diagram of a variation rule of a first gain upper limit according to an exemplary embodiment.

[0059] Figure 4 is a schematic diagram of a variation rule of a second gain upper limit according to an exemplary embodiment.

[0060] Figure 5 is a block diagram of a device sound effect processing apparatus according to an exemplary embodiment.

[0061] Figure 6 is a block diagram of an electronic device 600 according to an exemplary embodiment. DETAILED DESCRIPTION

[0062] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0063] It should be noted that all actions of obtaining signals, information, or data in the present disclosure are carried out on the premise of complying with the corresponding data protection regulations and policies of the country where the location is located and obtaining authorization from the owner of the corresponding device.

[0064] The HRTF (Head-Related Transfer Function) is crucial in spatial audio. By controlling the level and phase differences between the ears, the HRTF helps users determine the direction of sound. Based on the HRTF loaded in the audio device, the recognition accuracy of spatialization for users is improved. For example, by creating Dolby Atmos spatial audio files, users can experience panoramic sound effects and enhance their cognitive accuracy of the stereo space. The device processes stereo files and creates an immersive virtual auditory experience based on the spatial audio engine.

[0065] The panoramic sound effect technology is applied to audio devices, including TWS (True Wireless Stereo) headphones, wired headphones, terminal devices, PC audio systems, hearing aids, etc., to help users enhance their spatial perception ability of the virtual space in corresponding scenarios based on the audio signals output by the device in multiple application scenarios such as MR (Mixed Reality scene), VR (Virtual Reality) / AR (Augmented Reality) environments, and games.

[0066] When spatial audio is applied to game scenarios to improve users' cognitive ability of the virtual space in game scenarios, in related technologies, when the same sound signal is input to different playback devices, the sound heard by users is different. If the loudness of the same game sound effect output by different terminals is inconsistent, it will damage the game experience. For example, in a shooting game, players generally need to judge the position of opponents through footsteps and vehicle sounds. If the loudness of the same footsteps output on different terminals is uneven, some players may not be able to judge the position of opponents in time, which will bring an extremely bad experience to players. By obtaining the game sound effect to be output, and determining whether the game sound effect needs to be processed according to the parameter configuration information of the terminal. If processing is required, call the target sound effecter to process the game sound effect, keep the original loudness unchanged, and output the processed game sound effect to avoid the problem of uneven sound effects caused when the same audio signal is output to different devices.

[0067] Exemplarily, it can also be used to solve problems such as how to achieve natural transition and matching of sound effects when a user continuously releases the same game skill in a game scenario, such as the shooting skill of a virtual firearm, so that players can feel the synchronization and coherence between the game sound effects and their game operations; how to play the last game sound effect in a manner that meets the player's expectations when stopping the release of the game skill to provide more satisfactory audio feedback; how to effectively manage and control the sound effects corresponding to multiple game skills in the game to ensure that they can be accurately played according to the frequency and context of game operations, thereby improving the quality and immersion of game sound effects. By responding to the release instruction of continuously releasing the same game skill, controlling the sound engine to loop-play the game sound effects in the sound sequence, where the sound sequence includes the game sound effects corresponding to multiple game skills, and the playback frequency is consistent with the release frequency. Responding to the stop instruction of stopping the release of the game skill, playing the last game sound effect triggered by the complete release instruction to solve the above problems, thereby improving the immersive experience of users in the game.

[0068] In the application process of spatial sound effects, based on the device's determination of the direction of the sound source in the virtual space, the determination of the far and near positions is particularly important. It can enable users to have a more realistic spatial experience by distinguishing the sense of direction of the sound source, so as to help users complete corresponding tasks. For example, in the application scenario of a shooting game with spatial sound effects, users can lock the position of the sound source in the virtual game scenario based on the directions of the sound sources of footsteps, firearm shooting sounds, and firearm reloading sounds, thereby simulating the real scenario and assisting users in making judgments on the current game situation. In the related technologies, none of them can solve the problem that when the device plays an audio signal, the user has a poor perception of the sound effect localization rendered by the device engine. At this time, the user cannot judge the sound source direction of the sound effect in the virtual scenario, and thus cannot react in time, which destroys the user's usage experience.

[0069] In view of this, the present application proposes a device sound effect processing method, device, electronic device, and storage medium. By analyzing and enhancing these sound signals, players can more accurately judge the source and direction of the sound, thereby providing a more immersive game experience. At the same time, it also helps players better locate and identify the positions of other players and game elements in complex multiplayer interactions. This technology is particularly useful in team cooperation and competitive games and can improve players' strategies and reaction abilities.

[0070] Figure 1 is a flowchart of a device sound effect processing method shown according to an exemplary embodiment. As Figure 1 shown, this method is used in a terminal and includes the following steps.

[0071] In step S11, in response to receiving a first-channel input signal and a second-channel input signal, determine a first energy value of the first-channel input signal and determine a second energy value of the second-channel input signal.

[0072] Exemplarily, this embodiment is applied to a terminal, which may be an auditory device such as TWS (True Wireless Stereo) headphones, wired headphones, a terminal device, a PC audio system, and a hearing aid. The terminal includes multiple channel input interfaces. Based on these channel input interfaces, an upper-layer audio input device of the terminal can input a first-channel input signal and a second-channel input signal to the terminal, where the audio input device may be a terminal such as a PC terminal, a mobile phone, a laptop, and a tablet computer for outputting audio data content outward. The multiple channel input interfaces in the terminal correspond to multiple channel output ports of the terminal. For example, based on the setting of the human binaural, when the terminal performs sound effect output, a left-channel output and a right-channel output will be set. Correspondingly, the multiple channel input interfaces include a left-channel input interface and a right-channel input interface, and the first-channel input signal and the second-channel input signal include a left-channel input signal and a right-channel input signal.

[0073] Optionally, in some embodiments, the first-channel input signal is a left-channel input signal, and the second-channel input signal is a right-channel input signal.

[0074] Exemplarily, in this embodiment, the first-channel input signal may be a left-channel input signal, and the second-channel input signal may be a right-channel input signal. Exemplarily, in some embodiments, the first-channel input signal may be a right-channel input signal, and the second-channel input signal may be a left-channel input signal. This embodiment does not make any limitations in this regard.

[0075] Exemplarily, in some embodiments, the channel input interfaces of the terminal may include more than 3, which can be set based on different application scenarios of the terminal. For example, when the terminal is a concert sound effect device, based on the characteristics of a large concert venue and a large number of corresponding users, in order to enable each user to have a more immersive usage experience, multiple channel input interfaces can be set corresponding to multiple channel output interfaces to assist users in having a more immersive sound effect experience.

[0076] When the terminal receives the first-channel input signal and the second-channel input signal sent by the upper-layer input audio input device, it calculates the first energy value of the first-channel input signal and the second energy value of the second-channel input signal through the energy calculation formula of audio energy. For example, in some embodiments, during the process of calculating the energy value, each frame of the input signal is used as the processing unit, and the input signal is processed frame by frame. The received channel input signal is discretely sampled to obtain the channel input signal of multiple sampling points, and then the energy values of each channel input signal are weighted and calculated to generate the energy value of the channel input signal.

[0077] For example, in some embodiments, this embodiment proposes a calculation method for the short-time energy value of the channel input signal, as shown in the following formula:

[0078]

[0079] Where P i is the first energy value or the second energy value, N is the number of sampling points in the channel input signal, n is the serial number of the discrete sampling point, and S i (n) is the channel input signal of the nth sampling point. By weighting and summing the energy values of the N discrete sampling points in the channel input signal as described above, the energy value of the channel input signal is obtained.

[0080] Optionally, in some embodiments, the first energy value is the first short-time energy value, the second energy value is the second short-time energy value, and the target energy value is the target short-time energy value.

[0081] For example, in this embodiment, the voice frame is used as the processing unit of the channel input signal, and the corresponding determined energy value is the short-time energy value of the channel input signal. By analyzing this short-time energy value, the azimuth relationship of the current channel input signal in the virtual display space is determined.

[0082] In this embodiment, the energy values of different channel input signals are analyzed to determine the differences between each channel input signal. If the differences are large, the channel input signal with a larger energy value can be enhanced, so that the user can clearly perceive the differences between different channel input signals based on the terminal, and also simulate different sound source positions in the virtual scene. For example, in the virtual scene, an explosion occurs on the right side of the user, generating a large audio input, while there is no other additional sound source on the left side generating an audio input signal. Then, when the energy value of the left-channel input signal received by the left-channel interface of the terminal is less than the energy value of the right-channel input signal received by the right-channel interface, according to the comparison of the energy values, the right-channel input signal is enhanced, so as to widen the difference between the left-channel and right-channel input signals, so that the user can experience the sound source azimuth of the explosion coming from the right hand direction of the current azimuth based on the channel input signal of the terminal, giving the user a more realistic sound source azimuth experience.

[0083] Optionally, in some embodiments, the above step S11 includes:

[0084] Obtain the first historical energy value of the first-channel input signal in the previous frame and the first forgetting factor of the channel input signal;

[0085] Determine the first initial energy value of the first-channel input signal;

[0086] Perform autoregressive smoothing filtering on the first initial energy value according to the first historical energy value and the first forgetting factor to generate the first energy value;

[0087] Obtain the second historical energy value of the second-channel input signal in the previous frame;

[0088] Determine the second initial energy value of the second-channel input signal;

[0089] Perform autoregressive smoothing filtering on the second initial energy value according to the second historical energy value and the first forgetting factor to generate the second energy value.

[0090] Exemplarily, in this embodiment, autoregressive smoothing filtering is performed on the first energy value of the first-channel input signal and the second energy value of the second-channel input signal to smooth the energy value based on the waveform characteristics of the input signal and obtain a more stable processing result. Obtain the first historical energy value of the first-channel input signal in the previous frame and the first forgetting factor of the channel input signal. The method for determining the first initial energy value of the first-channel input signal is the same as the above method, and the above calculation method can be referred to and will not be elaborated here. Perform autoregressive smoothing filtering on the first initial energy value according to the first forgetting factor and the first historical energy value, thereby generating the first energy value of the first-channel input signal. Exemplarily, the first energy value can be calculated by the following formula:

[0091]

[0092] where P i is the first energy value, N is the number of sampling points in the channel input signal, n is the serial number of the discrete sampling point, s i (n) is the channel input signal of the nth sampling point, P 0 is the first historical energy value, α is the first forgetting factor, and the first initial energy value is calculated and determined according to the formula . Among them, to avoid the influence of abrupt interference signals on the energy value, the value range of α in this embodiment is [0.1, 0.2], and the first energy value after smoothing filtering is calculated according to the above formula. In this embodiment, taking the first energy value of the first-channel input signal as an example, the processing method of the corresponding second energy value of the second-channel input signal is the same as that of the first energy value and will not be elaborated here.

[0093] Optionally, in some embodiments, the method further includes:

[0094] Obtain a first initial channel input signal and a second initial channel input signal;

[0095] Filter the first initial channel input signal and the second initial channel input signal according to a preset filtering frequency band to generate a first channel input signal and a second channel input signal.

[0096] Exemplarily, in this embodiment, the channel input signals input from the upper-layer audio input device to the terminal include various types of sounds. For example, they include near-field audio signals, far-field audio signals, or background music signals, etc. Usually, the near-field audio signal is audio data in a virtual scene where the user can visually determine the specific sound source position. Therefore, during the process of enhancement processing determination, in order to avoid interference from other audio signals to the far-field audio signal, it is necessary to filter out the far-field audio signal emitted from a long-distance scene from the channel input signals, analyze this far-field audio signal, and determine whether it is necessary to enhance the audio signals of the first channel input signal or the second input signal. In this embodiment, the audio signals transmitted by the upper-layer audio input device are filtered according to a preset filtering frequency band to generate a filtered first channel input signal and a second channel input signal. Then, the first channel input signal and the second channel input signal are determined in the following manner to determine the channel input signal that needs to be enhanced.

[0097] Optionally, in some embodiments, the preset filtering frequency band is 0 - 1000 Hz or 0 - 2000 Hz.

[0098] Exemplarily, the preset filtering frequency band is set to [0 - 1000 Hz] or [0 - 2000 Hz], and the terminal filters out the channel input signals of [0 - 1000 Hz] or [0 - 2000 Hz] from the initial channel input signals based on this preset filtering frequency band, and calculates the energy values of the filtered channel input signals.

[0099] Optionally, in some embodiments, the above "determine the first energy value of the first channel input signal and determine the second energy value of the second channel input signal" includes:

[0100] Obtain the device sound effect mode;

[0101] When it is determined that the device sound effect mode is the game scene mode, determine the first energy value and the second energy value.

[0102] For example, the processing method of the channel input signal corresponding to the terminal is different in different application scenarios. For example, when the terminal is used to play data signals such as music and videos, the user does not need to confirm the direction of the corresponding sound source based on the audio signal. Therefore, at this time, it is not necessary to perform gain processing on the channel input signal; when the terminal is used to receive audio data signals in a game scenario, in order to provide the user with a more immersive gaming experience, it is necessary to perform different gain processing on the channel input signal based on the direction of the sound source in the game scenario. In this embodiment, before determining the energy value of the channel input signal, the device sound effect mode corresponding to the terminal set by the user in the upper-layer audio input device is obtained. When it is determined that the current is the game scenario mode, the first energy value of the first channel input signal and the second energy value of the second input signal are determined. Among them, the device sound effect mode may include a music sound effect mode, a bass enhancement mode, a clear voice mode, a game sound effect mode, etc.

[0103] In step S12, according to the first energy value and the second energy value, determine the relative energy ratio of the first channel signal and the second channel signal.

[0104] For example, in this embodiment, the relative energy ratio between the first channel signal and the second channel signal is determined by relative ratio calculation. For example, in this embodiment, a calculation method for the relative energy ratio is proposed:

[0105]

[0106] where r is the relative energy ratio, P 1 is the first energy value, P 2 is the second energy value. Based on the above formula, the relative energy ratio between the first channel signal and the second channel signal is determined, and thus the difference between the first channel signal and the second channel signal is determined based on the r value.

[0107] Optionally, in some embodiments, the above step S12 includes:

[0108] Obtain the historical relative energy ratio of the previous frame of audio signal and the second forgetting factor of the target channel signal;

[0109] According to the first energy value and the second energy value, determine the initial relative energy ratio;

[0110] According to the historical relative energy ratio and the second forgetting factor, perform autoregressive smoothing filtering on the initial relative energy ratio to generate the relative energy ratio.

[0111] Exemplarily, usually the channel input signal is a waveform signal. Based on the smoothness of waveform signal processing, in the process of determining the relative energy ratio based on the waveform signal, autoregressive smoothing filtering needs to be performed on the relative energy ratio of the currently processed channel input signal based on the historical relative energy ratio of the previous frame of audio signal, so as to suppress or remove the interference of noise on the relative energy ratio and make the data more stable and predictable. In this embodiment, the historical relative energy ratio of the previous frame of audio signal and the second forgetting factor of the target channel signal are obtained. After determining the initial relative energy ratio of the first energy value and the second energy value based on the above method, autoregressive smoothing filtering is performed on the initial relative energy ratio according to the second forgetting factor and the historical relative energy ratio to generate the relative energy ratio. Among them, to avoid the influence of abrupt signals on the relative energy ratio in the currently processed channel input signal, the value of the second forgetting factor can be set to be relatively small. For example, the value range of the second forgetting factor is [0.1 - 0.2], so that the relative energy ratio is close to the historical relative energy ratio of the previous frame of audio signal, and a smooth relative energy ratio result is obtained.

[0112] Exemplarily, the relative energy ratio is calculated by the following formula:

[0113]

[0114] where r is the relative energy ratio, P 1 is the first energy value, P 2 is the second energy value, β is the second forgetting factor, and r 0 is the historical relative energy ratio of the previous frame of audio signal. By β, the weight of the most recent observation value is determined, making the relative energy ratio smoother, and a more accurate relative energy ratio is obtained based on the stability and smoothness of audio signal processing.

[0115] In step S13, according to the target energy value and the relative energy ratio, the gain parameter of the target channel signal is determined, where the target channel signal is the channel signal with a larger energy value among the first channel signal and the second channel signal, and the target energy value is the energy value of the target channel signal.

[0116] Exemplarily, compare the magnitudes of the first energy value and the second energy value, and determine the larger one as the target energy value. The channel signal corresponding to the target energy value is the target channel signal. To increase the difference between the first channel input signal and the second channel input signal, it is necessary to perform gain on the channel input signal with a larger energy value, so that the user can feel an obvious channel difference based on the terminal. The gain parameter can be determined based on the relative energy ratio and the target energy value. For example, if the relative energy ratio is large and the target energy value is low, a larger gain parameter can be set to increase the difference between the first channel input signal and the second channel input signal; if the relative energy ratio is small, it means that the difference between the first channel input signal and the second channel input signal is small, which means that the sound source distance of the first channel input signal is close to that of the second channel input signal. Therefore, a smaller gain parameter can be set or the gain parameter can be set to 0. In this embodiment, the target energy value and the relative energy ratio generated in the above steps can be input into the gain calculation model, and the gain parameter of the target channel signal can be determined based on this gain calculation model.

[0117] In step S14, perform equalization filtering on the target channel signal based on the gain parameter to output the target audio signal.

[0118] Exemplarily, after determining the gain parameter of the target channel input signal in the above manner, perform equalization filtering on the target channel input signal based on this gain parameter to generate the target audio signal. For example, by the above method, it is determined that the first energy value of the first channel input signal is greater than the second energy value of the second channel input signal, and the first channel input signal is determined as the target channel input signal, and the first energy value is the target energy value. After determining the gain parameter of the first channel input signal according to the above method, perform gain on the first channel input signal, and use the gain-adjusted first channel input signal and the second channel input signal as the target audio signal, and play the gain-adjusted first channel input signal based on the first channel output interface of the terminal, and play the second channel input signal based on the second channel output interface of the terminal.

[0119] It should be noted that in this embodiment, equalization filtering is performed on the target channel signal based on the gain parameter. Equalization filtering is a signal processing technique used to adjust the frequency response of a signal to improve the sound equalization and enhance the sound characteristics within a specific frequency range. Equalization filtering is used in audio processing. By adjusting the volume of the audio within a specific frequency range, the sound of different frequencies can be emphasized or weakened, thereby improving the listening effect of the audio. Exemplarily, after determining the frequency range of the long-distance audio signal under normal circumstances in this embodiment, first filter the audio signal within this frequency range in the target channel input signal, and then perform gain on the filtered audio signal based on the gain parameter to obtain the gain-adjusted target channel signal.

[0120] In the above manner, in response to receiving the first-channel input signal and the second-channel input signal, the first energy value of the first-channel input signal is determined and the second energy value of the second-channel input signal is determined. According to the first energy value and the second energy value, the relative energy ratio of the first-channel signal and the second-channel signal is determined. According to the target energy value and the relative energy ratio, the gain parameter of the target-channel signal is determined, where the target-channel signal is the channel signal with the larger energy value among the first-channel signal and the second-channel signal, and the target energy value is the energy value of the target-channel signal. Based on the gain parameter, equalization filtering is performed on the target-channel signal to output the target audio signal. Thus, the calculation of the energy value of the channel input signal and the ratio calculation are combined, and then mapped to the gain adjustment of the channel, enhancing the direction positioning ability of the channel input signal in the terminal and providing a more accurate and immersive auditory experience for the user.

[0121] Figure 2 is a flowchart of a method for determining a gain parameter shown according to an exemplary embodiment. As Figure 2 shown, this method is applied to a terminal, and the above step S13 includes the following steps.

[0122] In step S131, if the relative energy ratio is within a preset range, then according to the relative energy ratio, the first gain upper limit of the target-channel signal is determined, and according to the target energy value, the second gain upper limit of the target-channel signal is determined.

[0123] Exemplarily, in this embodiment, the value range of the relative energy ratio is [0, 1]. The closer the relative energy ratio is to 0, it indicates that the first energy value and the second energy value are the same, and the audio loudness of the corresponding first-channel input signal and the second-channel input signal is the same; the closer the relative energy ratio is to 1, it indicates that the difference between the first-channel input signal and the second input signal is larger. At this time, there is no need to perform gain on the input signal, and the user can clearly distinguish the first-channel input signal and the second-channel input signal from the audio signal. Therefore, for the channel input signals with smaller differences and larger differences, in this embodiment, no gain processing is performed on the channel input signals, and the corresponding gain value can be set to 0.

[0124] In this embodiment, the variation law between the relative energy ratio and the first gain upper limit corresponding to the target-channel signal is determined through finite experiments, and the variation law curve between the first gain upper limit and the relative energy ratio is obtained. Exemplarily, Figure 3 is a schematic diagram of the variation law of a first gain upper limit shown according to an exemplary embodiment. As Figure 3 shown, according to this variation law curve, the transformation function of the first gain upper limit is obtained, and based on this transformation function, the first gain upper limit corresponding to the current relative energy ratio is determined.

[0125] Optionally, in some embodiments, the above step of "determining the first gain upper limit of the target channel signal according to the relative energy ratio" includes:

[0126] Obtain the first maximum gain parameter of the target channel signal;

[0127] Determine the first gain upper limit according to the first maximum gain parameter and the relative energy ratio.

[0128] Exemplarily, in this embodiment, considering the maximum gain and human comfort, the first maximum gain parameter of the target channel signal is determined through limited experiments, and then the first gain upper limit is calculated based on the first maximum gain parameter and the relative energy ratio.

[0129] Optionally, in some embodiments, the above step of "determining the first gain upper limit according to the first maximum gain parameter and the relative energy ratio" includes:

[0130]

[0131] Where gain1 is the first gain upper limit, r is the relative energy ratio, and maxGain1 is the first maximum gain parameter.

[0132] Exemplarily, in this embodiment, by analyzing the above variation law curve, the corresponding relationship between the first maximum gain parameter and the relative energy ratio is determined, and the calculation formula of the above first gain upper limit is determined based on this corresponding relationship, and the first gain upper limit is obtained through this formula. Among them, the value of maxGain1 can be 3.

[0133] Exemplarily, in this embodiment, the second gain upper limit corresponding to the gain parameter is determined based on the target energy value of the target channel input signal. It should be noted that if the target energy value is large, in order to avoid sound quality distortion caused by audio signal enhancement, the channel input signal with a large energy value is not subjected to gain processing.

[0134] Optionally, in some embodiments, the above step of "determining the second gain upper limit of the target channel signal according to the target energy value" includes:

[0135] Obtain the second maximum gain parameter of the target channel signal;

[0136] Determine the second gain upper limit according to the second maximum gain parameter and the target energy value.

[0137] Exemplarily, in this embodiment, the second gain upper limit corresponding to the current target energy value is determined according to the variation law of the target energy value and the second gain upper limit. Figure 4 It is a schematic diagram of the variation law of the second gain upper limit shown according to an exemplary embodiment, such as Figure 4As shown, determine the variation function of the energy value and the second gain upper limit according to this variation rule:

[0138]

[0139] Where gain2 is the second gain upper limit, maxGain2 is the second maximum gain parameter, p is the target energy value, a is the preset slope parameter, and b is the preset constant. Determine the second gain upper limit of the target energy value through the above calculation formula. For example, the value of maxGain2 can be 7.

[0140] In step S132, determine the gain parameter according to the first gain upper limit and the second gain upper limit.

[0141] For example, multiply the first gain upper limit by the second gain upper limit to obtain the gain parameter of the target channel input signal.

[0142] In step S133, if the relative energy ratio is not within the preset interval range, determine the gain parameter as the preset gain parameter.

[0143] For example, based on the variation rule between the relative energy ratio and the first gain upper limit, it can be determined that the preset interval range is [0.2, 0.8]. If the relative energy ratio is within this preset interval range, determine the gain parameter based on the first gain upper limit and the second gain upper limit; if the relative energy ratio is not within the range of [0.2, 0.8], that is, r < 0.2 or r > 0.8, the gain parameter is the preset gain parameter, where the preset gain parameter can be set to 0.

[0144] Through the above method, determine the gain parameter of the target energy value based on the calculation methods of the first gain upper limit and the second gain upper limit, so as to avoid sound distortion caused by excessive gain while ensuring the difference between the first channel input signal and the second channel input signal after gain, and obtain a reasonable and scientific gain parameter.

[0145] Figure 5 It is a block diagram of a device sound effect processing device shown according to an exemplary embodiment. Refer to Figure 5 , the device 100 includes: a first determination module 110, a second determination module 120, a third determination module 130, and an execution module 140.

[0146] The first determination module 110 is configured to determine the first energy value of the first channel input signal and determine the second energy value of the second channel input signal in response to receiving the first channel input signal and the second channel input signal;

[0147] The second determination module 120 is configured to determine the relative energy ratio of the first channel signal and the second channel signal according to the first energy value and the second energy value;

[0148] A third determination module 130, configured to determine a gain parameter of a target channel signal according to a target energy value and an energy relative ratio, where the target channel signal is the channel signal with a larger energy value among a first channel signal and a second channel signal, and the target energy value is the energy value of the target channel signal;

[0149] An execution module 140, configured to perform equalization filtering on the target channel signal based on the gain parameter to output a target audio signal.

[0150] Optionally, the third determination module 130 includes:

[0151] A first determination sub-module, configured to, if the energy relative ratio is within a preset interval range, determine a first gain upper limit of the target channel signal according to the energy relative ratio, and determine a second gain upper limit of the target channel signal according to the target energy value;

[0152] A determination sub-module, configured to determine the gain parameter according to the first gain upper limit and the second gain upper limit;

[0153] A second determination sub-module, configured to, if the energy relative ratio is not within the preset interval range, determine that the gain parameter is a preset gain parameter.

[0154] Optionally, the first determination sub-module includes:

[0155] An acquisition unit, configured to acquire a first maximum gain parameter of the target channel signal;

[0156] A determination unit, configured to determine the first gain upper limit according to the first maximum gain parameter and the energy relative ratio.

[0157] Optionally, the determination unit is configured to:

[0158] Determine the first gain upper limit through the following formula;

[0159]

[0160] where gain1 is the first gain upper limit, r is the energy relative ratio, and maxGain1 is the first maximum gain parameter.

[0161] Optionally, the first determination sub-module is configured to:

[0162] Acquire a second maximum gain parameter of the target channel signal;

[0163] Determine the second gain upper limit according to the second maximum gain parameter and the target energy value.

[0164] Optionally, the first determination module 110 is configured to:

[0165] Obtain the first historical energy value of the first-channel input signal of the previous frame and the first forgetting factor of the channel input signal;

[0166] Determine the first initial energy value of the first-channel input signal;

[0167] Perform autoregressive smoothing filtering on the first initial energy value according to the first historical energy value and the first forgetting factor to generate the first energy value;

[0168] Obtain the second historical energy value of the second-channel input signal of the previous frame;

[0169] Determine the second initial energy value of the second-channel input signal;

[0170] Perform autoregressive smoothing filtering on the second initial energy value according to the second historical energy value and the first forgetting factor to generate the second energy value.

[0171] Optionally, the second determination module 120 is configured to:

[0172] Obtain the historical energy relative ratio of the audio signal of the previous frame and the second forgetting factor of the target channel signal;

[0173] Determine the initial energy relative ratio according to the first energy value and the second energy value;

[0174] Perform autoregressive smoothing filtering on the initial energy relative ratio according to the historical energy relative ratio and the second forgetting factor to generate the energy relative ratio.

[0175] Optionally, the apparatus 100 further includes a generation module, and the generation module is configured to:

[0176] Obtain the first initial channel input signal and the second initial channel input signal;

[0177] Filter the first initial channel input signal and the second initial channel input signal according to a preset filtering frequency band to generate the first-channel input signal and the second-channel input signal.

[0178] Optionally, the preset filtering frequency band is 0 - 1000 Hz or 0 - 2000 Hz

[0179] Optionally, the first determination module 110 is configured to:

[0180] Obtain the device sound effect mode;

[0181] When it is determined that the device sound effect mode is the game scene mode, determine the first energy value and the second energy value.

[0182] Optionally, the first channel input signal is a left channel input signal, the second channel input signal is a right channel input signal, and / or,

[0183] The first energy value is a first short-time energy value, the second energy value is a second short-time energy value, and the target energy value is a target short-time energy value.

[0184] Regarding the device in the above embodiments, the specific manner in which each module performs operations has been described in detail in the embodiments related to the method, and will not be elaborated here.

[0185] The present disclosure also provides a computer-readable storage medium, on which computer program instructions are stored, and when the program instructions are executed by a processor, the steps of the device sound effect processing method provided by the present disclosure are implemented.

[0186] Figure 6 FIG. 600 is a block diagram of an electronic device 600 shown according to an exemplary embodiment. For example, the electronic device 600 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0187] Refer to Figure 6 , the electronic device 600 may include one or more of the following components: a processing component 602, a memory 604, a power supply component 606, a multimedia component 606, an audio component 610, an input / output interface 612, a sensor component 614, and a communication component 616.

[0188] The processing component 602 generally controls the overall operation of the electronic device 600, such as operations associated with display, telephone call, data communication, camera operation, and recording operation. The processing component 602 may include one or more processors 620 to execute instructions to complete all or part of the steps of the above device sound effect processing method. In addition, the processing component 602 may include one or more modules to facilitate the interaction between the processing component 602 and other components. For example, the processing component 602 may include a multimedia module to facilitate the interaction between the multimedia component 606 and the processing component 602.

[0189] The memory 604 is configured to store various types of data to support the operation of the electronic device 600. Examples of such data include instructions for any application or method operating on the electronic device 600, contact data, phone book data, messages, pictures, videos, and the like. The memory 604 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.

[0190] The power supply component 606 provides power to various components of the electronic device 600. The power supply component 606 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 600.

[0191] The multimedia component 606 includes a screen that provides an output interface between the electronic device 600 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can not only sense the boundaries of the touch or swipe actions, but also detect the duration and pressure associated with the touch or swipe operation. In some embodiments, the multimedia component 606 includes a front camera and / or a rear camera. When the electronic device 600 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.

[0192] The audio component 610 is configured to output and / or input audio signals. For example, the audio component 610 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 600 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 604 or transmitted via the communication component 616. In some embodiments, the audio component 610 further includes a speaker for outputting audio signals.

[0193] The input / output interface 612 provides an interface between the processing component 602 and a peripheral interface module, which can be a keyboard, a click wheel, buttons, etc. These buttons can include, but are not limited to: a home button, a volume button, a power button, and a lock button.

[0194] The sensor assembly 614 includes one or more sensors for providing an assessment of the status of various aspects of the electronic device 600. For example, the sensor assembly 614 can detect the on / off state of the electronic device 600, the relative positioning of components, such as the display and keypad of the electronic device 600. The sensor assembly 614 can also detect a change in the position of the electronic device 600 or a component of the electronic device 600, the presence or absence of user contact with the electronic device 600, the orientation or acceleration / deceleration of the electronic device 600, and a change in the temperature of the electronic device 600. The sensor assembly 614 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 614 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 614 can further include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0195] The communication component 616 is configured to facilitate communication between the electronic device 600 and other devices in a wired or wireless manner. The electronic device 600 can access a wireless network based on communication standards, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component 616 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 616 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0196] In an exemplary embodiment, the electronic device 600 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above-described device sound effect processing method.

[0197] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 604 including instructions, and the above instructions can be executed by a processor 620 of the electronic device 600 to complete the above-described device sound effect processing method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.

[0198] In addition to being an independent electronic device, the above device can also be a part of an independent electronic device. For example, in one embodiment, the device can be an integrated circuit (IC) or a chip. The integrated circuit can be a single IC or a collection of multiple ICs. The chip can include, but is not limited to, the following types: GPU (Graphics Processing Unit), CPU (Central Processing Unit), FPGA (Field Programmable Gate Array), DSP (Digital Signal Processor), ASIC (Application Specific Integrated Circuit), SOC (System on Chip), etc. The above integrated circuit or chip can be used to execute executable instructions (or code) to implement the above device sound effect processing method. The executable instructions can be stored in the integrated circuit or chip, or obtained from other devices or equipment. For example, the integrated circuit or chip includes a processor, a memory, and an interface for communicating with other devices. The executable instructions can be stored in the memory, and when the executable instructions are executed by the processor, the above device sound effect processing method is implemented; or, the integrated circuit or chip can receive the executable instructions through the interface and transmit them to the processor for execution to implement the above device sound effect processing method.

[0199] In another exemplary embodiment, a computer program product is also provided. The computer program product includes a computer program that can be executed by a programmable device. The computer program has a code portion for executing the above device sound effect processing method when executed by the programmable device.

[0200] Those skilled in the art will readily think of other embodiments of the present disclosure after considering the specification and practicing the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0201] It should be understood that the present disclosure is not limited to the exact structures already described and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for processing device sound effects, characterized in that, it includes: In response to receiving a first-channel input signal and a second-channel input signal, determining a first energy value of the first-channel input signal and determining a second energy value of the second-channel input signal; According to the first energy value and the second energy value, determining an energy relative ratio of the first-channel signal and the second-channel signal; According to a target energy value and the energy relative ratio, determining a gain parameter of a target channel signal, where the target channel signal is the channel signal with a larger energy value among the first-channel signal and the second-channel signal, and the target energy value is the energy value of the target channel signal; Performing equalization filtering on the target channel signal based on the gain parameter to output a target audio signal.

2. The method according to claim 1, characterized in that, The determining the gain parameter of the target channel signal according to the target energy value and the energy relative ratio includes: If the energy relative ratio is within a preset interval range, determining a first gain upper limit of the target channel signal according to the energy relative ratio, and determining a second gain upper limit of the target channel signal according to the target energy value; Determining the gain parameter according to the first gain upper limit and the second gain upper limit; If the energy relative ratio is not within the preset interval range, determining the gain parameter as a preset gain parameter.

3. The method according to claim 2, characterized in that, The determining the first gain upper limit of the target channel signal according to the energy relative ratio includes: Obtaining a first maximum gain parameter of the target channel signal; Determining the first gain upper limit according to the first maximum gain parameter and the energy relative ratio.

4. The method according to claim 3, characterized in that, The determining the first gain upper limit according to the first maximum gain parameter and the energy relative ratio includes: Determining the first gain upper limit through the following formula; where the gain1 is the first gain upper limit, the r is the energy relative ratio, and the maxGain1 is the first maximum gain parameter.

5. The method according to claim 2, characterized in that, The determining the second gain upper limit of the target channel signal according to the target energy value includes: Obtaining a second maximum gain parameter of the target channel signal; Determining the second gain upper limit according to the second maximum gain parameter and the target energy value.

6. The method according to any one of claims 1-5, characterized in that, The determining the first energy value of the first-channel input signal and determining the second energy value of the second-channel input signal includes: Obtaining a first historical energy value of the first-channel input signal in the previous frame and a first forgetting factor of the channel input signal; Determining a first initial energy value of the first-channel input signal; Performing autoregressive smoothing filtering on the first initial energy value according to the first historical energy value and the first forgetting factor to generate the first energy value; Obtain the second historical energy value of the second-channel input signal of the previous frame; Determine the second initial energy value of the second-channel input signal; Perform autoregressive smoothing filtering on the second initial energy value according to the second historical energy value and the first forgetting factor to generate the second energy value.

7. The method according to any one of claims 1-5, wherein, the determining the relative energy ratio of the first-channel input signal and the second-channel input signal according to the first energy value and the second energy value includes: Obtain the historical relative energy ratio of the audio signal of the previous frame and the second forgetting factor of the target-channel signal; Determine the initial relative energy ratio according to the first energy value and the second energy value; Perform autoregressive smoothing filtering on the initial relative energy ratio according to the historical relative energy ratio and the second forgetting factor to generate the relative energy ratio.

8. The method according to any one of claims 1-5, wherein, the method further includes: Obtain the first initial-channel input signal and the second initial-channel input signal; Filter the first initial-channel input signal and the second initial-channel input signal according to a preset filtering frequency band to generate the first-channel input signal and the second-channel input signal.

9. The method according to claim 8, wherein, the preset filtering frequency band is 0-1000 Hz or 0-2000 Hz.

10. The method according to any one of claims 1-5, wherein, the determining the first energy value of the first-channel input signal and the second energy value of the second-channel input signal includes: Obtain the device sound effect mode; When it is determined that the device sound effect mode is the game scene mode, determine the first energy value and the second energy value.

11. The method according to any one of claims 1-5, wherein, the first-channel input signal is the left-channel input signal, the second-channel input signal is the right-channel input signal, and / or, the first energy value is the first short-time energy value, the second energy value is the second short-time energy value, and the target energy value is the target short-time energy value.

12. A device sound effect processing device, wherein, the device includes: A first determination module, configured to determine the first energy value of the first-channel input signal and the second energy value of the second-channel input signal in response to receiving the first-channel input signal and the second-channel input signal; A second determination module, configured to determine the relative energy ratio of the first-channel signal and the second-channel signal according to the first energy value and the second energy value; A third determination module, configured to determine the gain parameter of the target-channel signal according to the target energy value and the relative energy ratio, wherein the target-channel signal is the channel signal with a larger energy value among the first-channel signal and the second-channel signal, and the target energy value is the energy value of the target-channel signal; An execution module, configured to perform equalization filtering on the target channel signal based on the gain parameter to output a target audio signal.

13. An electronic device, characterized in that, it includes: a memory on which a computer program is stored; a processor, configured to execute the computer program in the memory to implement the steps of the device sound effect processing method according to any one of claims 1-11.

14. A computer-readable storage medium, on which computer program instructions are stored, characterized in that, when the program instructions are executed by a processor, the steps of the device sound effect processing method according to any one of claims 1-11 are implemented.

Images