Audio processing method and device and terminal equipment

Abstract

The invention provides an audio processing method and apparatus, and a terminal device. The method comprises the steps of obtaining a first audio and a distortion parameter associated with the first audio; performing up-sampling processing on the first audio to obtain an up-sampled audio; based on the distortion parameter, performing distortion processing on the up-sampling audio to obtain a second audio; performing down-sampling processing on the second audio to obtain a down-sampled audio; and determining a distorted audio associated with the first audio based on the first audio and the down-sampled audio.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of audio processing technology, and in particular, to an audio processing method, apparatus, and terminal device. Background Art

[0002] Distortion sound effect is a kind of audio playback effect. Distortion sound effect can be used to modulate rock music, thus improving the playback effect of music.

[0003] At present, the terminal device can perform distortion processing on the audio based on the hardware device so that the audio has a distorted sound effect. For example, the terminal device can adjust the distorted sound effect of the audio based on the sound effect adjustment device so that the audio has a better distorted sound effect. However, the accuracy of the distortion processing of the audio based on the above method is poor. Summary of the invention

[0004] The present disclosure provides an audio processing method, apparatus and terminal device, which are used to solve one or more technical problems in the prior art.

[0005] In a first aspect, the present disclosure provides an audio processing method, the audio processing method comprising:

[0006] Acquire a first audio and a distortion parameter associated with the first audio;

[0007] Performing up-sampling processing on the first audio to obtain up-sampled audio;

[0008] Based on the distortion parameter, performing distortion processing on the up-sampled audio to obtain a second audio;

[0009] Downsampling the second audio to obtain downsampled audio;

[0010] Based on the first audio and the downsampled audio, distorted audio associated with the first audio is determined.

[0011] In a second aspect, the present disclosure provides an audio processing device, the audio processing device comprising an acquisition module, an upsampling module, a distortion module, a downsampling module and a determination module, wherein:

[0012] The acquisition module is used to acquire the first audio and the distortion parameter associated with the first audio;

[0013] The up-sampling module is used to perform up-sampling processing on the first audio to obtain up-sampled audio;

[0014] The distortion module is used to perform distortion processing on the up-sampled audio based on the distortion parameter to obtain a second audio;

[0015] The down-sampling module is used to perform down-sampling processing on the second audio to obtain down-sampled audio;

[0016] The determination module is used to determine the distorted audio associated with the first audio based on the first audio and the down-sampled audio.

[0017] In a third aspect, an embodiment of the present disclosure provides a terminal device including: a processor and a memory;

[0018] The memory stores computer-executable instructions;

[0019] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the audio processing method as described in the first aspect and various possible aspects of the first aspect.

[0020] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the audio processing method as described in the first aspect and various possible aspects of the first aspect are implemented.

[0021] The present disclosure provides an audio processing method, apparatus and terminal device. The terminal device can obtain a first audio and a distortion parameter associated with the first audio, and perform up-sampling processing on the first audio to obtain an up-sampled audio. The terminal device can perform distortion processing on the up-sampled audio based on the distortion parameter to obtain a second audio, and perform down-sampling processing on the second audio to obtain a down-sampled audio. The terminal device can determine the distorted audio associated with the first audio based on the first audio and the down-sampled audio. In the above method, since the terminal device can perform up-sampling processing on the first audio and perform distortion processing on the up-sampled audio, the accuracy of the distortion processing performed by the terminal device on the audio is relatively high, thereby improving the accuracy of the distortion processing and the accuracy of the distorted sound effect of the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0023] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0024] Figure 2 A flowchart of an audio processing method provided by an embodiment of the present disclosure;

[0025] Figure 3 A schematic diagram of a process for obtaining a first audio provided by an embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram of a process for determining upsampled audio provided by an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of a result after distortion processing provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of a process for determining upsampled audio provided by an embodiment of the present disclosure;

[0029] Figure 7 A schematic diagram of a method for determining distorted audio provided by an embodiment of the present disclosure;

[0030] Figure 8 A schematic diagram of the structure of an audio processing device provided by an embodiment of the present disclosure; and

[0031] Figure 9 A schematic diagram of the structure of a terminal device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0033] To facilitate understanding, the concepts involved in the embodiments of the present disclosure are explained below.

[0034] Terminal device: a device with wireless transceiver function. The terminal device can be deployed on land, including indoors or outdoors, handheld, wearable or vehicle-mounted. The terminal device can be a mobile phone, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in industrial control, a vehicle-mounted terminal device, a wireless terminal in self-driving, a wireless terminal device in remote medical, a wireless terminal device in smart grid, a wireless terminal device in transportation safety, a wireless terminal device in smart city, a wireless terminal device in smart home, a wearable terminal device, etc. The terminal device involved in the embodiments of the present disclosure can also be called a terminal, a user equipment (UE), an access terminal device, a vehicle-mounted terminal, an industrial control terminal, a UE unit, a UE station, a mobile station, a mobile station, a remote station, a remote terminal device, a mobile device, a UE terminal device, a wireless communication device, a UE agent or a UE device, etc. Terminal devices can also be fixed or mobile.

[0035] In the related art, distortion sound effect is a kind of audio playback effect. Distortion sound effect can be used for the modulation of rock music. Based on distortion sound effect, the playback effect of rock music can be improved. At present, the terminal device can perform distortion processing on the audio based on the hardware device, so that the audio has the distortion sound effect. For example, the terminal device can adjust the distortion sound effect of the audio based on the sound effect adjustment device, but in this way, the terminal device needs to set the hardware sound effect adjuster, resulting in a high adjustment cost of the distortion sound effect of the terminal device, and the current distortion sound effect adjustment is mainly for the distortion sound effect of instruments such as guitars and drums, and the distortion processing effect for human voices and humming audio is poor, and the adjustment method of the distortion sound effect in the related art is to directly modify the audio so that the audio has the distortion sound effect, but the sampling rate of the audio is usually small, which leads to low accuracy of distortion processing and poor playback effect of the distortion sound effect.

[0036] In order to solve the technical problems in the related technology, the embodiment of the present disclosure provides an audio processing method, the terminal device can obtain the first audio and the distortion parameters associated with the first audio, wherein the distortion parameters may include the distortion type and the distortion degree, the terminal device can up-sample the first audio to obtain the up-sampled audio, and determine the distortion function corresponding to the distortion type, the terminal device can, based on the distortion function and the distortion degree, perform distortion processing on the first sampling value of each sampling point in the up-sampled audio to obtain the second sampling value after distortion of each sampling point, determine the second audio based on multiple second sampling values ​​associated with multiple sampling points, the terminal device can down-sample the second audio to obtain the down-sampled audio, and determine the distorted audio associated with the first audio based on the first audio and the down-sampled audio. In the above method, the terminal device does not need to use a hardware-based audio adjustment device to adjust the distorted sound effects of the audio, thereby reducing the cost of adjusting the distorted sound effects. In addition, since the audio used is the audio after upsampling the first audio, the sampling frequency of the upsampled audio is higher than the sampling frequency of the first audio. Therefore, the terminal device can accurately perform distortion processing on the upsampled audio, thereby improving the accuracy of the distortion processing, and thereby improving the playback effect of the distorted audio associated with the first audio.

[0037] Next, combine Figure 1 , the application scenarios of the embodiments of the present disclosure are described.

[0038] Figure 1 A schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 1 , including: a terminal device. Among them, the display page of the terminal device can be a distortion page. The distortion page can include music to be processed and a distortion parameter setting page. Among them, in the distortion parameter setting page, the distortion type is type 1 (you can select other distortion types after clicking the subscript arrow), the distortion degree is 0.355, the sampling rate is 8 times sampling (you can select other sampling rates after clicking the subscript arrow), the mixing ratio is 0.831, and the gain after distortion is -5.996. When the terminal device plays music, the terminal device can control the distortion sound effect of the music based on the pre-set distortion parameters.

[0039] Please refer to the figure. During the music playback, the user can adjust the value of any parameter in the distortion parameters, and the terminal device can adjust the distorted sound effect of the music in real time. In this way, the terminal device can adjust the distorted sound effect of the music based on software, reducing the adjustment cost of the distorted sound effect. Moreover, since the terminal device can perform distortion processing after upsampling the audio, the accuracy of the distortion processing is high, which can improve the accuracy of the distorted sound effect of the music. Moreover, since the user can flexibly adjust the distortion parameters during the music playback, the flexibility of adjusting the distorted sound effect can be improved.

[0040] 1. Figure 1 The above is only an example of the application scenario of the embodiment of the present disclosure, and is not a limitation of the application scenario of the embodiment of the present disclosure.

[0041] The technical solution of the present invention and how the technical solution of the present invention solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present invention will be described below in conjunction with the accompanying drawings.

[0042] Figure 2 A flowchart of an audio processing method provided by an embodiment of the present disclosure. Figure 2 , the method may include:

[0043] S201. Obtain a first audio and a distortion parameter associated with the first audio.

[0044] The execution subject of the embodiment of the present disclosure may be a terminal device, or an audio processing device arranged in the terminal device. The audio processing device may be implemented based on software, or may be implemented based on a combination of software and hardware, which is not limited in the embodiment of the present disclosure.

[0045] The first audio may be audio to be distorted. For example, the terminal device may distort audio 1, and the terminal device may determine audio 1 as the first audio; the terminal device may distort audio 2, and the terminal device may determine audio 2 as the first audio.

[0046] Optionally, the terminal device can obtain the first audio based on the following feasible implementation method: display an audio processing page, wherein the audio processing page includes an audio processing area and an audio file, and in response to a move operation on the audio file, move the audio file to the audio processing area to obtain the first audio.

[0047] The audio processing page may include an audio processing area and an audio file area, and the audio file area may include multiple audio files pre-stored by the terminal device. For example, if the terminal device pre-stores audio 1, audio 2, and audio 3, when the terminal device displays the audio processing page, the audio file area in the audio processing page may include the files of audio 1, audio 2, and audio 3.

[0048] The first audio is associated with an audio file. For example, the audio processing page may include audio file 1 and audio file 2. The terminal device may move audio file 1 to the audio processing area in response to a user's move operation on audio file 1, and the terminal device may determine the audio associated with audio file 1 as the first audio. The terminal device may move audio file 2 to the audio processing area in response to a user's move operation on audio file 2, and the terminal device may determine the audio associated with audio file 2 as the first audio.

[0049] Next, combine Figure 3 , a process of obtaining the first audio by the terminal device is described.

[0050] Figure 3 A schematic diagram of a process for obtaining a first audio provided by an embodiment of the present disclosure. Figure 3 , including: a terminal device. The display page of the terminal device may be an audio processing page, and the audio processing page may include audio file 1, audio file 2, audio file 3, and an audio processing area. The user may long press audio file 2 and move audio file 2 to the audio processing area. After the user releases audio file 2, the terminal device may determine the audio related to audio file 2 as the first audio. In this way, the user may flexibly select the audio to be distorted, thereby improving the flexibility of obtaining the first audio.

[0051] It should be noted that Figure 3 This is only an example of the audio processing page, and is not a limitation on the elements, layout, and other information in the audio processing page.

[0052] It should be noted that the terminal device can obtain the first audio based on any other feasible implementation method (such as receiving the first audio sent by other devices, or obtaining the first audio from a database, or recording the first audio in real time, etc.), and the embodiments of the present disclosure are not limited to this.

[0053] Optionally, the distortion parameter may include a distortion type and a distortion degree. The distortion type may indicate a distortion mode of the audio. For example, the distortion type of the audio may be a function-type distortion and a regular-type distortion, wherein the function-type distortion may include a cubic distortion and a hyperbolic tangent distortion, and the regular-type distortion may include an L1 regular-type distortion and an L2 regular-type distortion.

[0054] It should be noted that the distortion type in the embodiment of the present disclosure may also be any other achievable distortion type, and the embodiment of the present disclosure is not limited to this.

[0055] The distortion degree may be a distortion degree associated with the distortion type. For example, if the distortion type is function-type distortion, the distortion degree may be a distortion degree corresponding to the function-type distortion, that is, a degree of distortion of the original dry sound (dry sound when the audio is not distorted, and wet sound after the audio is distorted).

[0056] It should be noted that the terminal device may display a sound effect adjustment page, which may include a control control associated with the distortion parameter, and the terminal device may determine the distortion parameter associated with the first audio in response to the user's operation on the control control. Figure 1 In the embodiment shown, the user can select the distortion type in the distortion parameter setting page (the terminal device can determine the distortion type corresponding to the field based on the field related to the distortion type), or set the distortion degree in the distortion parameter setting page, which is not limited in the embodiment of the present disclosure.

[0057] It should be noted that the terminal device may also determine the distortion type and the distortion degree based on any other feasible implementation manner (eg, presetting the distortion type and the distortion degree), and the embodiments of the present disclosure are not limited to this.

[0058] S202: Perform up-sampling processing on the first audio to obtain up-sampled audio.

[0059] The upsampled audio may be audio obtained by upsampling the first audio. For example, if the sampling rate of the first audio is 44k, the terminal device may upsample the first audio to obtain the upsampled audio, wherein the sampling rate of the upsampled audio may be 192k.

[0060] Optionally, the terminal device can obtain upsampled audio based on the following feasible implementation methods: determine the upsampling rate, upsample the first audio based on the upsampling rate to obtain the first audio to be processed, and filter the first audio to be processed based on the first filter to obtain the upsampled audio.

[0061] The upsampling rate may indicate the upsampling ratio of the first audio. For example, if the upsampling rate is 8, the terminal device may upsample the first audio by 8 times, and if the upsampling rate is 32, the terminal device may upsample the first audio by 32 times.

[0062] Optionally, the terminal device may determine the upsampling rate based on the user's operation. Figure 1 In the illustrated embodiment, the distortion parameter setting page may include a sampling rate, which is 8 times. Therefore, the terminal device may determine that the up-sampling rate is 8.

[0063] It should be noted that the terminal device can determine the upsampling rate based on any feasible implementation method (such as receiving the upsampling rate sent by other devices, or pre-setting a fixed upsampling rate, etc.), and the embodiments of the present disclosure are not limited to this.

[0064] Among them, the first filter is used to filter the first audio to be processed. The first audio to be processed may be the audio after the first audio is upsampled. However, since the image frequency will be introduced after upsampling, the image frequency will exist in the first audio to be processed. The terminal device can filter the image frequency in the first audio to be processed based on the first filter, and then obtain the upsampled audio. The sampling frequency of the upsampled audio is greater than the sampling frequency of the first audio, and there is no image frequency in the upsampled audio.

[0065] It should be noted that the first filter may be any filter that filters the image frequency, and the embodiments of the present disclosure are not limited to this.

[0066] Next, combine Figure 4 , the process of determining the upsampled audio by the terminal device is described.

[0067] Figure 4 A schematic diagram of a process for determining upsampled audio provided by an embodiment of the present disclosure. Figure 4 , including a first audio, an upsampling module and a first filter. The upsampling module and the first filter can be set in a terminal device ( Figure 4 The sampling frequency of the first audio may be 44000, and the terminal device may input the first audio and the upsampling rate to the upsampling module, and the upsampling module may upsample the first audio based on the upsampling rate to obtain the first audio to be processed, wherein the sampling frequency of the first audio to be processed is 192000, and the first audio to be processed has a mirror frequency.

[0068] See also Figure 4 , the terminal device can input the first audio to be processed into the first filter, and the first filter can filter the image frequency in the first audio to be processed, and then output the upsampled audio, wherein the sampling frequency of the upsampled audio can be 192000. In this way, the terminal device can obtain the upsampled audio with higher sampling accuracy based on the upsampling module and the first filter, and then improve the accuracy of the distortion processing of the upsampled audio, thereby improving the accuracy of the distortion processing of the first audio, and improving the effect of the distortion sound effect of the first audio.

[0069] S203: Perform distortion processing on the upsampled audio based on the distortion parameter to obtain a second audio.

[0070] The second audio may be audio after distortion processing of the upsampled audio. For example, the upsampled audio is audio obtained by upsampling the first audio, so after the distortion processing of the upsampled audio, the second audio obtained may be associated with the first audio.

[0071] Optionally, the terminal device may obtain the second audio based on the following feasible implementation manner: determining a distortion function corresponding to the distortion type, and performing distortion processing on the upsampled audio based on the distortion function and the distortion degree to obtain the second audio.

[0072] Among them, the distortion function can be used to perform distortion processing on the audio. For example, the audio may include multiple sampling points, each sampling point corresponds to a sampling value, and the terminal device can perform distortion processing on the sampling value of each sampling point based on the distortion function, and then perform distortion processing on the audio to obtain audio with distorted sound effects. In this way, the terminal device can perform distortion processing on the first audio based on flexible selection of distortion type, thereby improving the flexibility of distortion processing and improving the user experience.

[0073] The distortion function may correspond to the distortion type. For example, if the distortion type is type 1, then type 1 may correspond to distortion function A; if the distortion type is type 2, then type 2 may correspond to distortion function 2; and if the distortion type is type 3, then type 3 may correspond to distortion function 3.

[0074] Optionally, the distortion type may include at least one of the following: cubic distortion, hyperbolic tangent distortion, L1 regularized distortion, and L2 regularized distortion.

[0075] Among them, the distortion function associated with the cubic distortion can be:

[0076]

[0077] Among them, the distortion function associated with the hyperbolic tangent distortion can be:

[0078] f(x)=tanh(x)

[0079] Among them, the distortion function associated with L1 regularized distortion can be:

[0080]

[0081] Among them, the distortion function associated with L2 regularized distortion can be:

[0082]

[0083] Among them, f(x) in the above formula can be the sampling value after the sampling point distortion processing, x can be the sampling value of the sampling point, and if the waveform of the first audio signal is sin(x×π / 2), then the value range of x in the above formula can be [-1.0, 1.0].

[0084] Next, combine Figure 5 , the results of distortion processing of the first audio by multiple distortion functions are described.

[0085] Figure 5 This is a schematic diagram of a distortion processing result provided by an embodiment of the present disclosure. Figure 5 , including a coordinate system, wherein the horizontal axis of the coordinate system may be a sampling value x before the sampling point is distorted, and the vertical axis of the coordinate system may be a sampling value y after the sampling point is distorted. The coordinate system may include a curve of the original audio, a curve of L1 distortion (L1 regularized distortion), a curve of L2 distortion (L2 regularized distortion), and an interval of hyperbolic tangent distortion.

[0086] Optionally, the terminal device performs distortion processing on the upsampled audio based on the distortion function and the distortion degree to obtain the second audio. Specifically, the first sampling value of each sampling point in the upsampled audio is distorted based on the distortion function and the distortion degree to obtain the distorted second sampling value of each sampling point, and the second audio is determined based on multiple second sampling values ​​associated with multiple sampling points. In this way, since there are a large number of sampling points in the upsampled audio, the terminal device can accurately perform distortion processing on the upsampled audio, thereby improving the accuracy of the distortion processing of the first audio.

[0087] The first sampling value may be a sampling value of the first audio at a sampling point. For example, if the sampling value of the first audio at sampling point 1 is 0.2, then the first sampling value of sampling point 1 is 0.2; if the sampling value of the first audio at sampling point 2 is 0.4, then the first sampling value of sampling point 2 is 0.2.

[0088] The second sampling value may be a value after the first sampling value is distorted. For example, for any sampling point, if the first sampling value of the sampling point in the first audio is 0.2, the terminal device processes the first sampling value based on the distortion function and the distortion degree, and the obtained sampling value is 0.18. The terminal device may determine that the second sampling value of the sampling point is 0.18.

[0089] Among them, the terminal device can perform distortion processing on the first sampling value of the sampling point based on a distortion function. For example, if the distortion type is cubic distortion, for any sampling point, the first sampling value of this sampling point in the first audio is 0.2. The terminal device can obtain the corresponding x of 0.2 and substitute x into the distortion function of cubic distortion to obtain f(x). The terminal device can determine f(x) as the second sampling value of this sampling point.

[0090] It should be noted that during the process of the terminal device performing distortion processing on the first sampling value, the distortion degree can be used to limit the degree of distortion. For example, the first sampling value of the sampling point is 0.2. After the terminal device performs distortion processing on the first sampling value based on the distortion function, the second sampling value of this sampling point is 0.18. If the distortion degree is 0.01, the distortion degree of this sampling point (distortion of 0.02) is greater than the distortion degree, and the terminal device can set the second sampling value to 0.19. If the distortion degree is 0.03, the distortion degree of this sampling point is less than the distortion degree, and the terminal device can determine the second sampling value to be 0.18.

[0091] It should be noted that the terminal device can also, based on any feasible implementation method, perform distortion processing on the first sampling value of each sampling point in the upsampled audio based on the distortion function and the distortion degree. The embodiments of the present disclosure do not limit this.

[0092] Optionally, after the terminal device determines multiple second sampling values associated with multiple sampling points, it can determine a second audio based on the multiple second sampling values.

[0093] S204. Perform downsampling processing on the second audio to obtain a downsampled audio.

[0094] Among them, the downsampled audio can be the audio after performing downsampling processing on the second audio. For example, if the sampling rate of the second audio is 192k, the terminal device can perform downsampling processing on the second audio to obtain a downsampled audio, where the sampling rate of the downsampled audio can be 44k.

[0095] It should be noted that the sampling rate of the downsampled audio can be the same as the sampling rate of the first audio.

[0096] Optionally, the terminal device can obtain the downsampled audio based on the following feasible implementation method: determine the downsampling rate associated with the upsampling rate, perform downsampling processing on the second audio based on the downsampling rate to obtain a second audio to be processed, and perform filtering processing on the second audio to be processed based on a second filter to obtain the downsampled audio.

[0097] The downsampling rate may indicate a multiple of downsampling the second audio. For example, if the downsampling rate is 8, the terminal device may downsample the second audio by 8 times, and if the downsampling rate is 32, the terminal device may downsample the second audio by 32 times.

[0098] It should be noted that after the terminal device determines the upsampling rate, the sampling rate of the downsampling rate is the same as the sampling rate of the upsampling rate. For example, if the upsampling rate is 8 times the upsampling rate, the downsampling rate is 8 times the downsampling rate, and if the upsampling rate is 32 times the upsampling rate, the downsampling rate is 32 times the downsampling rate.

[0099] Among them, the second filter is used to filter the second audio to be processed. The second audio to be processed can be the audio after the second audio is downsampled. However, since aliasing frequency will be introduced after downsampling, aliasing frequency will exist in the second audio to be processed. The terminal device can filter the aliasing frequency in the second audio to be processed based on the second filter, and then obtain the downsampled audio. The sampling frequency of the downsampled audio is the same as the sampling frequency of the first audio, and there is no aliasing frequency in the downsampled audio.

[0100] It should be noted that the second filter may be any filter that filters the aliasing frequency, and the embodiments of the present disclosure are not limited to this.

[0101] Next, combine Figure 6 , the process of determining the downsampled audio by the terminal device is described.

[0102] Figure 6 A schematic diagram of a process for determining upsampled audio provided by an embodiment of the present disclosure. Figure 6 , including a second audio, a downsampling module and a second filter. The downsampling module and the second filter can be set in the terminal device ( Figure 6 The sampling frequency of the second audio may be 192000, and the terminal device may input the second audio and the downsampling rate to the downsampling module, and the downsampling module may downsample the second audio based on the downsampling rate to obtain a second audio to be processed, wherein the sampling frequency of the second audio to be processed is 444000, and there is an aliasing frequency in the second audio to be processed.

[0103] See also Figure 6, the terminal device can input the second audio to be processed into the second filter, and the second filter can filter the aliasing frequency in the second audio to be processed, and then output the downsampled audio, wherein the sampling frequency of the downsampled audio can be 44000. In this way, the terminal device can restore the sampling frequency of the second audio based on the downsampling module and the second filter, and since the distortion accuracy of the second audio is high, the distortion accuracy of the downsampled audio is high, thereby improving the distortion accuracy of the first audio.

[0104] S205: Determine, based on the first audio and the downsampled audio, a distorted audio associated with the first audio.

[0105] The distorted audio may be audio that is distorted from the first audio, and the distorted audio may include distorted sound effects. Optionally, the terminal device may combine the first audio and the downsampled audio to obtain the distorted audio associated with the first audio.

[0106] Optionally, after the terminal device determines the distorted audio, it can obtain the post-distortion gain. The terminal device can adjust the overall volume of the distorted audio based on the post-distortion gain, thereby obtaining a continuously adjustable loudness experience, which can improve the user experience and improve the playback effect of the distorted audio.

[0107] The embodiment of the present disclosure provides an audio processing method, the terminal device can obtain a first audio and a distortion parameter associated with the first audio, the terminal device can determine an upsampling rate, and upsample the first audio based on the upsampling rate to obtain a first audio to be processed, filter the first audio to be processed based on a first filter to obtain an upsampled audio, the terminal device can determine a distortion function corresponding to a distortion type, the terminal device can perform distortion processing on a first sampling value of each sampling point in the upsampled audio based on the distortion function and the distortion degree to obtain a second sampling value of each sampling point after distortion, determine a second audio based on multiple second sampling values ​​associated with multiple sampling points, the terminal device determines a downsampling rate associated with the upsampling rate, and downsamples the second audio based on the downsampling rate to obtain a second audio to be processed, filter the second audio to be processed based on the second filter to obtain a downsampled audio screen, and the terminal device can determine the distorted audio associated with the first audio based on the first audio and the downsampled audio. In the above method, the terminal device does not need to use a hardware-based audio adjustment device to adjust the distorted sound effects of the audio, thereby reducing the cost of adjusting the distorted sound effects. In addition, the terminal device can filter the mirror frequency introduced by upsampling based on the first filter, and filter the aliasing frequency introduced by downsampling based on the second filter. Moreover, since the upper audio is the audio after upsampling the first audio, the sampling frequency of the upsampled audio is higher than the sampling frequency of the first audio. Therefore, the terminal device can accurately perform distortion processing on the upsampled audio, improve the accuracy of the distortion processing, and thereby improve the playback effect of the distorted audio associated with the first audio.

[0108] exist Figure 2 Based on the embodiment shown below, combined with Figure 7 , a method for determining the distorted audio associated with the first audio based on the first audio and the down-sampled audio in the above audio processing method is described in detail.

[0109] Figure 7 A schematic diagram of a method for determining distorted audio provided by an embodiment of the present disclosure. Figure 7 The method flow may include:

[0110] S701: Determine a mixing ratio between a first audio and a down-sampled audio.

[0111] Optionally, the mixing ratio may be a dry-sound / wet-sound mixing ratio. For example, the first audio is the audio before distortion processing, so the first audio may be a dry audio, and the downsampled audio is the audio after distortion processing, so the downsampled audio may be a wet audio. The terminal device may combine the first audio and the downsampled audio based on the dry-sound / wet-sound mixing ratio to obtain the distorted audio.

[0112] Optionally, the terminal device may determine the mixing ratio between the first audio and the downsampled audio based on user operation. Figure 1 In the illustrated embodiment, the distortion parameter setting page may include a control of a mixing ratio, and the user may adjust the mixing ratio between the first audio and the downsampled audio based on an operation on the control.

[0113] It should be noted that the terminal device can determine the mixing ratio between the first audio and the downsampled audio based on any feasible implementation method (such as based on a pre-set mixing ratio, etc.), and the embodiments of the present disclosure are not limited to this.

[0114] S702: Based on the mixing ratio, the first audio and the down-sampled audio are combined to obtain distorted audio.

[0115] Optionally, since the mixing ratio can indicate the ratio of dry sound to wet sound in the distorted audio, the terminal device can combine the sampling values ​​of each sampling point in the first audio and the downsampled audio based on the mixing ratio. For example, if the mixing ratio is 0.7, 70% of the sampling value of each sampling point in the distorted audio is determined by the wet sound and 30% by the dry sound.

[0116] Among them, the terminal device can perform mixing processing on the first audio and the downsampled audio based on the following feasible implementation method: for any target sampling point of the distorted audio, multiply the mixing ratio by the sampling value of the target sampling point of the downsampled audio to obtain a third sampling value, determine the difference between the preset value and the mixing ratio, and multiply the difference by the sampling value of the target sampling point of the first audio to obtain a fourth sampling value, and add the third sampling value and the fourth sampling value to obtain the target sampling value of the target sampling point. In this way, the terminal device can improve the playback effect of the distorted sound effect in the distorted audio and improve the user experience.

[0117] Optionally, the first audio and the downsampled audio also include the target sampling point. For example, since the first audio, the downsampled audio, and the distorted audio have the same sampling frequency, if the target sampling point is the first sampling point of the distorted audio, the target sampling point in the first audio may also be the first sampling point in the first audio, and the target sampling point in the downsampled audio may also be the first sampling point in the first audio.

[0118] The third sampling value may be the product of the mixing ratio and the sampling value of the target sampling point of the downsampled audio. For example, if the mixing ratio is 0.5 and the sampling value of the target sampling point in the downsampled audio is 0.8, the terminal device may determine that the third sampling value is 0.4.

[0119] The preset value may be 1, and the terminal device may determine the difference between 1 and the mixing ratio. For example, if the mixing ratio is 70%, the difference may be 30%, and if the mixing ratio is 90%, the difference may be 10%.

[0120] The fourth sampling value may be the product of the difference value and the sampling value of the target sampling point of the first audio. For example, if the mixing ratio is 0.6, the difference between the preset value and the mixing ratio is 0.4, and if the sampling value of the target sampling point in the downsampled audio is 0.8, the terminal device may determine that the fourth sampling value is 0.32.

[0121] Optionally, the terminal device may determine the sum of the third sampling value and the fourth sampling value as the target sampling value of the target sampling point. For example, if the mixing ratio is 0.6, the sampling value of the target sampling point in the downsampled audio is 0.4, and the sampling value of the target sampling point in the first audio is 0.5, then the third sampling value is 0.24, and the fourth sampling value is 0.2. The terminal device may determine that the target sampling value of the target sampling point is 0.44, that is, the sampling value of the target sampling point of the distorted audio is 0.44. The terminal device may determine the sampling value of each sampling point in the distorted audio based on the above method, and thus obtain the distorted audio.

[0122] The disclosed embodiment provides a method for determining distorted audio, wherein a terminal device can determine a mixing ratio between a first audio and a downsampled audio, and for any target sampling point of the distorted audio, multiply the mixing ratio by the sampling value of the target sampling point of the downsampled audio to obtain a third sampling value, determine a difference between a preset value and the mixing ratio, and multiply the difference by the sampling value of the target sampling point of the first audio to obtain a fourth sampling value, add the third sampling value and the fourth sampling value to obtain a target sampling value of the target sampling point, and the terminal device can obtain the distorted audio based on the target sampling value of each target sampling point in the distorted audio. In this way, the terminal device can mix the audio before the distortion processing with the audio after the distortion processing, thereby improving the playback effect of the distorted sound effect of the distorted audio, and thereby improving the user experience.

[0123] Figure 8 This is a structural diagram of an audio processing device provided by an embodiment of the present disclosure. Figure 8 , the audio processing device 800 includes an acquisition module 801, an up-sampling module 802, a distortion module 803, a down-sampling module 804 and a determination module 805, wherein:

[0124] The acquisition module 801 is used to acquire a first audio and a distortion parameter associated with the first audio;

[0125] The up-sampling module 802 is used to perform up-sampling processing on the first audio to obtain up-sampled audio;

[0126] The distortion module 803 is used to perform distortion processing on the up-sampled audio based on the distortion parameter to obtain a second audio;

[0127] The down-sampling module 804 is used to perform down-sampling processing on the second audio to obtain down-sampled audio;

[0128] The determination module 805 is configured to determine the distorted audio associated with the first audio based on the first audio and the down-sampled audio.

[0129] According to one or more embodiments of the present disclosure, the distortion module 803 is specifically used for:

[0130] Determining a distortion function corresponding to the distortion type;

[0131] Based on the distortion function and the distortion degree, distortion processing is performed on the up-sampled audio to obtain the second audio.

[0132] According to one or more embodiments of the present disclosure, the distortion module 803 is specifically used for:

[0133] Based on the distortion function and the distortion degree, a first sampling value of each sampling point in the upsampled audio is distorted to obtain a distorted second sampling value of each sampling point;

[0134] The second audio frequency is determined based on a plurality of second sampling values ​​associated with a plurality of sampling points.

[0135] According to one or more embodiments of the present disclosure, the up-sampling module 802 is specifically used for:

[0136] Determine the upsampling rate;

[0137] Performing upsampling processing on the first audio based on the upsampling rate to obtain a first audio to be processed;

[0138] The first audio to be processed is filtered based on a first filter to obtain the up-sampled audio.

[0139] According to one or more embodiments of the present disclosure, the downsampling module 804 is specifically used for:

[0140] determining a downsampling rate associated with the upsampling rate;

[0141] Downsampling the second audio based on the downsampling rate to obtain a second audio to be processed;

[0142] The second audio to be processed is filtered based on a second filter to obtain the downsampled audio.

[0143] According to one or more embodiments of the present disclosure, the determining module 805 is specifically configured to:

[0144] determining a mixing ratio between the first audio and the downsampled audio;

[0145] Based on the mixing ratio, the first audio and the down-sampled audio are combined to obtain the distorted audio.

[0146] According to one or more embodiments of the present disclosure, the determining module 805 is specifically configured to:

[0147] For any target sampling point of the distorted audio;

[0148] Multiplying the mixing ratio by the sampling value of the target sampling point of the down-sampled audio to obtain a third sampling value;

[0149] Determine a difference between a preset value and the mixing ratio, and multiply the difference by a sampling value of the target sampling point of the first audio to obtain a fourth sampling value;

[0150] The third sampling value and the fourth sampling value are added to obtain a target sampling value of the target sampling point.

[0151] According to one or more embodiments of the present disclosure, the acquisition module 801 is specifically used for:

[0152] Displaying an audio processing page, wherein the audio processing page includes an audio processing area and an audio file;

[0153] In response to a move operation on the audio file, the audio file is moved to the audio processing area to obtain the first audio, which is associated with the audio file.

[0154] The audio processing device provided in this embodiment can be used to execute the technical solution of the above method embodiment. Its implementation principle and technical effect are similar, and this embodiment will not be repeated here.

[0155] Figure 9 This is a schematic diagram of the structure of a terminal device provided by an embodiment of the present disclosure. Figure 9, which shows a schematic diagram of the structure of a terminal device 900 suitable for implementing the embodiment of the present disclosure. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 9 The terminal device shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0156] like Figure 9 As shown, the terminal device 900 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 to a random access memory (RAM) 903. Various programs and data required for the operation of the terminal device 900 are also stored in the RAM 903. The processing device 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0157] Typically, the following devices may be connected to the I / O interface 905: input devices 906 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 907 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 908 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 909. The communication devices 909 may allow the terminal device 900 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 9 The terminal device 900 is shown to have various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0158] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0159] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0160] The computer-readable medium may be included in the terminal device, or may exist independently without being installed in the terminal device.

[0161] The computer-readable medium carries one or more programs. When the one or more programs are executed by the terminal device, the terminal device executes the method shown in the above embodiment.

[0162] An embodiment of the present disclosure provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, various methods that may be involved in the above embodiments are implemented.

[0163] An embodiment of the present disclosure provides a computer program product, including a computer program, which implements various methods that may be involved in the above embodiments when executed by a processor.

[0164] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0165] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0166] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0167] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0168] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative and non-restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0169] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0170] It is understandable that before using the technical solutions disclosed in the embodiments of the present disclosure, the types, scope of use, usage scenarios, etc. of the personal information involved in the present disclosure should be informed to the user and the user's authorization should be obtained in an appropriate manner in accordance with relevant laws and regulations.

[0171] For example, in response to receiving an active request from a user, a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information. Thus, the user can independently choose whether to provide personal information to software or hardware such as a terminal device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure based on the prompt message. As an optional but non-limiting implementation method, in response to receiving an active request from a user, the method of sending a prompt message to the user can be, for example, a pop-up window, and the prompt message can be presented in text form in the pop-up window. In addition, the pop-up window can also carry a selection control for the user to choose "agree" or "disagree" to provide personal information to the terminal device.

[0172] It is understandable that the above notification and user authorization process are only illustrative and do not limit the implementation of the present disclosure. Other methods that meet relevant laws and regulations may also be applied to the implementation of the present disclosure. It is understandable that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws and regulations and relevant provisions. Data may include information, parameters, and messages, such as flow switching indication information.

[0173] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0174] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0175] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. An audio processing method, It is characterized in that include: Acquire a first audio and a distortion parameter associated with the first audio; Performing up-sampling processing on the first audio to obtain up-sampled audio; Based on the distortion parameter, performing distortion processing on the up-sampled audio to obtain a second audio; Downsampling the second audio to obtain downsampled audio; Based on the first audio and the downsampled audio, distorted audio associated with the first audio is determined.

2. The method according to claim 1, It is characterized in that The distortion parameter includes a distortion type and a distortion degree; and based on the distortion parameter, performing distortion processing on the upsampled audio to obtain a second audio includes: Determining a distortion function corresponding to the distortion type; Based on the distortion function and the distortion degree, distortion processing is performed on the up-sampled audio to obtain the second audio.

3. The method according to claim 2, It is characterized in that The step of performing distortion processing on the up-sampled audio based on the distortion function and the distortion degree to obtain the second audio includes: Based on the distortion function and the distortion degree, a first sampling value of each sampling point in the upsampled audio is distorted to obtain a distorted second sampling value of each sampling point; The second audio frequency is determined based on a plurality of second sampling values ​​associated with a plurality of sampling points.

4. The method according to any one of claims 1 to 3, It is characterized in that The upsampling the first audio to obtain the upsampled audio includes: Determine the upsampling rate; Performing upsampling processing on the first audio based on the upsampling rate to obtain a first audio to be processed; The first audio to be processed is filtered based on a first filter to obtain the up-sampled audio.

5. The method according to claim 4, It is characterized in that The downsampling the second audio to obtain the downsampled audio includes: determining a downsampling rate associated with the upsampling rate; Downsampling the second audio based on the downsampling rate to obtain a second audio to be processed; The second audio to be processed is filtered based on a second filter to obtain the downsampled audio.

6. The method according to any one of claims 1 to 3, It is characterized in that The determining, based on the first audio and the down-sampled audio, a distorted audio associated with the first audio, comprises: determining a mixing ratio between the first audio and the downsampled audio; Based on the mixing ratio, the first audio and the down-sampled audio are combined to obtain the distorted audio.

7. The method according to claim 6, It is characterized in that The merging the first audio and the down-sampled audio based on the mixing ratio includes: For any target sampling point of the distorted audio; Multiplying the mixing ratio by the sampling value of the target sampling point of the down-sampled audio to obtain a third sampling value; Determine a difference between a preset value and the mixing ratio, and multiply the difference by a sampling value of the target sampling point of the first audio to obtain a fourth sampling value; The third sampling value and the fourth sampling value are added to obtain a target sampling value of the target sampling point.

8. The method according to any one of claims 1 to 3, It is characterized in that The obtaining of the first audio includes: Displaying an audio processing page, wherein the audio processing page includes an audio processing area and an audio file; In response to a move operation on the audio file, the audio file is moved to the audio processing area to obtain the first audio, which is associated with the audio file.

9. An audio processing device, It is characterized in that It includes an acquisition module, an up-sampling module, a distortion module, a down-sampling module and a determination module, wherein: The acquisition module is used to acquire the first audio and the distortion parameter associated with the first audio; The up-sampling module is used to perform up-sampling processing on the first audio to obtain up-sampled audio; The distortion module is used to perform distortion processing on the up-sampled audio based on the distortion parameter to obtain a second audio; The down-sampling module is used to perform down-sampling processing on the second audio to obtain down-sampled audio; The determination module is used to determine the distorted audio associated with the first audio based on the first audio and the down-sampled audio.

10. A terminal device, It is characterized in that include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the audio processing method according to any one of claims 1 to 8.

11. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the audio processing method according to any one of claims 1 to 8 is implemented.

Images