Audio processing method, computer device and storage medium

Abstract

The application relates to an audio processing method, a computer device and a storage medium. By determining an effective frequency band of a target user, an effective frequency band part and a target frequency band part to be processed of target audio are obtained according to the effective frequency band, a target part audio signal corresponding to the target frequency band part is obtained, the target part audio signal is down-regulated to obtain a down-regulated target part audio signal, and an audio signal corresponding to the effective frequency band part of the target audio and the target part audio signal are synthesized to obtain target audio suitable for the target user. Compared with a traditional audio playing mode, the scheme combines the effective frequency band that can be heard by the target user, down-regulates the high-frequency part of the audio, and realizes the effect that the hearing-impaired user can hear the complete signal of the audio.

Description

Technical Field

[0001] This application relates to the field of audio processing technology, and in particular to an audio processing method, computer equipment, storage medium, and computer program product. Background Technology

[0002] Music currently focuses on the mid-to-high frequencies. However, people with hearing impairments can only hear low-frequency sound signals and cannot listen to music normally. Therefore, there is an urgent need for a way to process music audio so that the melody can be received by the ears of people with hearing impairments.

[0003] This means that current audio has the flaw of being unreceived by people with hearing impairments. Summary of the Invention

[0004] Therefore, it is necessary to provide an audio processing method, computer device, computer-readable storage medium, and computer program product that enables hearing-impaired individuals to receive audio signals, addressing the aforementioned technical problems.

[0005] In a first aspect, this application provides an audio processing method, the method comprising:

[0006] Obtain the target audio;

[0007] Identify the effective frequency bands for the target users;

[0008] Based on the effective frequency band, determine the effective frequency band portion of the target audio and the target frequency band portion to be processed;

[0009] Acquire the target portion audio signal corresponding to the target frequency band portion, and perform down-modulation processing on the target portion audio signal to obtain the down-modulated target portion audio signal;

[0010] Based on the synthesis result of the audio signal corresponding to the effective frequency band portion of the target audio and the target portion audio signal with reduced pitch, a target audio suitable for the target user is obtained.

[0011] In one embodiment, obtaining the effective frequency band portion and the target frequency band portion to be processed of the target audio based on the effective frequency band includes:

[0012] The target audio is downsampled to obtain the downsampled target audio;

[0013] Based on the effective frequency band, a wavelet transform is performed on the downsampled target audio to obtain the effective frequency band portion and the target frequency band portion to be processed.

[0014] In one embodiment, the step of performing wavelet transform on the downsampled target audio based on the effective frequency band to obtain the effective frequency band portion and the target frequency band portion to be processed includes:

[0015] Obtain the cutoff frequency corresponding to the frequency of the downsampled target audio;

[0016] The corresponding wavelet transform layer number is determined based on the effective frequency band and the cutoff frequency;

[0017] Using the cutoff frequency as the initial frequency, the target audio is subjected to wavelet transform according to the wavelet transform level and the initial frequency to obtain candidate low-frequency and candidate high-frequency components.

[0018] The candidate low-frequency portion containing the frequency corresponding to the effective frequency band is used as the new initial frequency. The step of performing wavelet transform on the target audio according to the wavelet transform level and the initial frequency is returned until the wavelet transform level is reached. The effective frequency band portion of the target audio is obtained according to the candidate low-frequency portion with the smallest frequency range, and the target frequency band portion to be processed is obtained according to at least one candidate high-frequency portion.

[0019] In one embodiment, down-pitching the target portion of the audio signal to obtain the down-pitched target portion of the audio signal includes:

[0020] For each target audio signal segment, obtain the wavelet transform level of the target frequency band segment corresponding to that target audio signal segment;

[0021] The corresponding pitch reduction amplitude is determined based on the wavelet transform level corresponding to the target audio signal, and the pitch of the target audio signal is reduced based on the pitch reduction amplitude to obtain the pitch-reduced target audio signal; the pitch reduction amplitude is negatively correlated with the wavelet transform level corresponding to the target audio signal.

[0022] In one embodiment, acquiring the target portion audio signal corresponding to the target frequency band portion includes:

[0023] The target frequency band portion is reconstructed using a multi-scale wavelet reconstruction function to obtain the target portion audio signal corresponding to the target frequency band portion.

[0024] In one embodiment, down-pitching the target portion of the audio signal to obtain the down-pitched target portion of the audio signal includes:

[0025] Obtain the percussion components and chord components from the target portion of the audio signal;

[0026] The percussion components are subjected to a first pitch reduction process based on a waveform similarity superposition algorithm, and the chord components are subjected to a second pitch reduction process based on a speed-variable pitch-invariant algorithm.

[0027] The target portion of the audio signal that has undergone the first and second down-pitch processing is upsampled to obtain the down-pitch target portion of the audio signal.

[0028] In one embodiment, after performing a first pitch reduction processing on the percussion component according to a waveform similarity superposition algorithm and a second pitch reduction processing on the chord component according to a speed-variable pitch-invariant algorithm, the method further includes:

[0029] If the difference in signal length between the target portion of the audio signal to be down-pitched and the target portion of the audio signal before down-pitching is greater than a preset threshold, the target portion of the audio signal is subjected to a third down-pitching process according to the variable speed unchanged pitch algorithm.

[0030] The target audio signal after the third down-pitch processing is upsampled to obtain the down-pitch target audio signal.

[0031] In one embodiment, obtaining the target audio suitable for the target user based on the synthesis result of the audio signal corresponding to the effective frequency band portion of the target audio and the down-pitch target portion audio signal includes:

[0032] The synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the target portion audio signal with reduced pitch is obtained;

[0033] The synthesized result is low-pass filtered, and the target audio suitable for the target user is obtained based on the low-pass filtered synthesized result.

[0034] Secondly, this application provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the method described above.

[0035] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the above-described method.

[0036] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0037] The aforementioned audio processing method, computer equipment, storage medium, and computer program product determine the effective frequency band for the target user, obtain the effective frequency band portion and the target frequency band portion to be processed from the effective frequency band, acquire the target portion audio signal corresponding to the target frequency band portion, down-pitch the target portion audio signal to obtain a down-pitch target portion audio signal, and synthesize the audio signal corresponding to the effective frequency band portion of the target audio and the target portion audio signal to obtain the target audio suitable for the target user. Compared to traditional audio playback methods, this solution, by combining the effective frequency band that the target user can hear and down-pitching the high-frequency part of the audio, achieves the effect of enabling hearing-impaired users to hear the complete audio signal. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an audio processing method in one embodiment;

[0039] Figure 2 This is a flowchart illustrating the wavelet transform steps in one embodiment;

[0040] Figure 3 This is a flowchart illustrating the pitch reduction step in one embodiment;

[0041] Figure 4 This is a flowchart illustrating the low-pass filtering steps in one embodiment;

[0042] Figure 5 This is a flowchart illustrating the audio processing method in another embodiment;

[0043] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0044] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0045] In one embodiment, such as Figure 1 As shown, an audio processing method is provided. This embodiment illustrates the method by applying it to a terminal. It is understood that the method can also be applied to a server, or to a system including both a terminal and a server, and is implemented through the interaction between the terminal and the server. The method includes the following steps:

[0046] Step S202: Obtain the target audio.

[0047] The target audio can be any audio that requires processing. For example, the target audio can be in the form of instrumental music. Since hearing-impaired individuals mostly only hear low-frequency sounds, the terminal can process the audio to create the processed target audio so that they can hear the full music. The target audio can be selected and input by the user of the terminal, or it can be automatically acquired by the terminal. For example, when the terminal detects instrumental music input, it can automatically acquire that instrumental music as the target audio for processing.

[0048] Step S204: Determine the effective frequency band for the target user.

[0049] The effective frequency band refers to the frequency band that the target user can hear, such as a frequency band below a certain frequency threshold. The terminal can determine the effective frequency band for the target user in various ways. For example, the terminal can count the frequency bands that multiple hearing-impaired individuals can hear, obtain multiple frequency bands, classify these multiple frequency bands to form multiple levels of candidate effective frequency bands, and then select one from these candidate effective frequency bands as the effective frequency band. Alternatively, in some embodiments, the terminal can also obtain the frequency band input by the target user, match the user-input frequency band with multiple pre-set effective frequency band levels, and determine the effective frequency band in which the target user is located.

[0050] Step S206: Determine the effective frequency band portion and the target frequency band portion to be processed based on the effective frequency band.

[0051] The effective frequency band can be a frequency range, such as a low-frequency range. The target audio also has a corresponding frequency range. The target audio may include the effective frequency band portion corresponding to the effective frequency band, and may also include the target frequency band portion to be processed. In some embodiments, the effective frequency band portion may be a low-frequency portion, and the target frequency band portion to be processed may be a high-frequency portion. That is, the terminal needs to perform relevant processing on the target frequency band portion to be processed.

[0052] Step S208: Obtain the target part audio signal corresponding to the target frequency band, and perform down-modulation processing on the target part audio signal to obtain the down-modulated target part audio signal.

[0053] The target frequency band mentioned above can be a frequency band determined from the transformed target audio. For example, the terminal performs a wavelet transform on the target audio to obtain the effective frequency band portion and the target frequency band portion to be processed. For each target frequency band portion, the terminal can acquire its corresponding target audio signal and down-pitch it to obtain a down-pitch target audio signal. The down-pitch target audio signal can be an audio signal with a reduced frequency, meaning the frequency band of the down-pitch target audio signal falls within the frequency range of the aforementioned effective frequency band.

[0054] After determining the target frequency band to be processed, the terminal needs to reconstruct the corresponding target audio signal within the target audio. For example, in one embodiment, the target frequency band to be processed can be obtained by the terminal performing wavelet transform on the target audio. The terminal can then reconstruct the audio of the target frequency band using a multi-scale wavelet reconstruction function to obtain the corresponding target audio signal.

[0055] Specifically, the target frequency band can be a high-frequency part. The terminal can perform audio reconstruction on all high-frequency parts after wavelet transform. For example, the terminal can use the waverec command in MATLAB, which is the multi-scale wavelet reconstruction function, to obtain the effective mid-to-high frequency signals that are only distributed in the target frequency band of the target audio, and use them as the target audio signal.

[0056] Step S210: Based on the synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the target audio signal with reduced pitch, a target audio suitable for the target user is obtained.

[0057] The effective frequency band of the target audio can be obtained after wavelet transform. To synthesize target audio suitable for the target user, the terminal can perform audio reconstruction on the effective part of the target audio to obtain the audio signal corresponding to the effective frequency band of the target audio. The terminal can combine the audio signal corresponding to the effective frequency band and the target audio signal with reduced pitch corresponding to the target frequency band. For example, the terminal synthesizes the audio signal corresponding to the effective frequency band of the target audio with the target audio signal with reduced pitch to obtain a corresponding synthesis result. Based on this synthesis result, a target audio suitable for the target user is obtained, meaning that the frequency band of the synthesized target audio is within the frequency range of the effective frequency band, allowing the target user to hear the complete sound in the target audio. The above synthesis can be performed based on signal superposition.

[0058] In the aforementioned audio processing method, the effective frequency band of the target user is determined. Based on the effective frequency band, the effective frequency band portion of the target audio and the target frequency band portion to be processed are obtained. The target portion audio signal corresponding to the target frequency band portion is obtained, and the pitch of the target portion audio signal is down-modulated to obtain the down-modulated target portion audio signal. Finally, by synthesizing the audio signal corresponding to the effective frequency band portion of the target audio and the target portion audio signal, the target audio suitable for the target user is obtained. Compared with traditional audio playback methods, this solution combines the effective frequency band that the target user can hear and down-modulates the high-frequency part of the audio, achieving the effect that hearing-impaired users can hear the complete audio signal.

[0059] In one embodiment, obtaining the effective frequency band portion and the target frequency band portion to be processed of the target audio based on the effective frequency band includes: downsampling the target audio to obtain downsampled target audio; and performing wavelet transform on the downsampled target audio based on the effective frequency band to obtain the effective frequency band portion and the target frequency band portion to be processed of the target audio.

[0060] In this embodiment, the effective frequency band portion of the target audio can be obtained through wavelet transform. The target audio corresponds to a specific frequency, and the terminal can downsample the target audio to obtain downsampled target audio. The downsampled target audio can be audio with a lower frequency compared to the original target audio. The wavelet transform can have different wavelet transform levels, also known as wavelet series, which represent the number of wavelet transform operations. A higher series results in a lower frequency after the transform. The terminal can perform wavelet transform on the downsampled target audio based on the effective frequency band to obtain the effective frequency band portion and the target frequency band portion to be processed. The effective frequency band portion of the target audio can be the portion of the target audio that falls within the effective frequency band range after wavelet transform, while the target frequency band portion to be processed can be the portion of the target audio that falls outside the effective frequency band range after wavelet transform. Since the effective frequency bands for different target users can be different and are all in the low frequency range, and the bandwidth of the low frequency bands corresponding to different wavelet transform levels is also different, the terminal can use the above effective frequency bands to determine the wavelet transform level, and then perform wavelet transform on the target audio based on the corresponding wavelet transform level.

[0061] For example, in one embodiment, the terminal can obtain the cutoff frequency corresponding to the frequency of the downsampled target audio, and determine the corresponding wavelet transform level based on the effective frequency band and the cutoff frequency. For instance, the terminal can obtain the corresponding wavelet transform level by inputting the effective frequency band and the cutoff frequency into a corresponding function. The cutoff frequency can be a specific frequency used to describe frequency characteristics. The terminal can use the cutoff frequency as the initial frequency, and perform a wavelet transform on the target audio based on the wavelet transform level and the initial frequency to obtain candidate low-frequency and candidate high-frequency components. The candidate low-frequency and candidate high-frequency components correspond to different frequency ranges, but the frequency range span of the candidate low-frequency component can be the same as that of the candidate high-frequency component.

[0062] The terminal can detect whether the number of wavelet transforms has reached the required wavelet transform level. If not, the terminal can use the candidate low-frequency portion containing the frequency corresponding to the effective frequency band as the new initial frequency, and return to the steps of performing wavelet transform on the target audio according to the wavelet transform level and the initial frequency. This process is repeated multiple times until the required wavelet transform level is reached. At this point, the terminal obtains a candidate low-frequency portion with the smallest frequency range and multiple candidate high-frequency portions with a larger frequency range than the candidate low-frequency portion. The terminal can obtain the effective frequency band portion of the target audio based on the candidate low-frequency portion with the smallest frequency range, and obtain the target frequency band portion to be processed based on at least one candidate high-frequency portion. That is, the terminal can process all audio in the target frequency band portion except for the effective frequency band portion.

[0063] Specifically, such as Figure 2 As shown, Figure 2This is a flowchart illustrating the wavelet transform steps in one embodiment. Taking instrumental music as an example, the terminal can downsample the input instrumental music to fs = 32kHz and determine its corresponding cutoff frequency. The specific function of the cutoff frequency can be as follows: fw = fs / 2 = 16kHz, where fw is the cutoff frequency. Different wavelet transform levels can represent different orders M. The terminal can adjust the frequency range of the effective frequency band after the wavelet transform, i.e., the effective low-frequency bandwidth, by modifying the order M. The larger M is, the narrower the corresponding low-frequency band, i.e., the narrower the bottom wavelet. For example, depending on the effective frequency band, M can be selected as levels 1, 2, 3, 4, or 5. The method for determining the wavelet transform level M based on the effective frequency band and the cutoff frequency can be as follows: fwl = fw / (2^M), where fwl represents the bandwidth of the bottom wavelet, i.e., the aforementioned effective frequency band. Taking a cutoff frequency of 16kHz as an example, the specific values ​​of the low-frequency bandwidth that can be obtained by M at each of the above levels can be: 8kHz, 4kHz, 2kHz, 1kHz, and 500Hz. In some embodiments, the terminal can use the db (Daubechies Wavelet) function with a boundary of 4 as the kernel function of the wavelet transform, that is, to perform the wavelet transform using the Daubechies wavelet function.

[0064] The terminal can perform db2 wavelet decomposition on the target audio signal with a cutoff frequency in MATLAB. This results in two vanishing momentum. Specifically, it is represented as: [c, l] = wavedec(x, M, 'db2'); where x represents the input pure music signal, M represents the number of wavelet layers to be decomposed (also called the series), c represents the final wavelet coefficients obtained after decomposition, and l represents the length information of each decomposition layer, also called the number of effective points corresponding to each coefficient segment; l can be presented in sequence form. Taking the effective frequency band of the target user as 4kHz as an example, the terminal can determine that the number of wavelet transform layers used is M = 2. Through wavelet transform, the following can be obtained: Figure 3 The results are shown. The terminal uses MATLAB to decompose the target audio with a cutoff frequency into candidate low-frequency and candidate high-frequency components in the first layer. The frequency band of the candidate low-frequency component is [0Hz, 8kHz], and the frequency band of the candidate high-frequency component is [8kHz, 16kHz]. The terminal can then perform wavelet transform on the candidate low-frequency component again to obtain the candidate low-frequency component [0Hz, 4kHz] and the candidate high-frequency component [4kHz, 8kHz] of the second layer of wavelet transform. At this point, the terminal can determine that the candidate low-frequency component [0Hz, 4kHz] is the effective frequency band of the target audio, and the frequency bands [4kHz, 8kHz] and [8kHz, 16kHz] are the target frequency bands to be processed.

[0065] Through the above embodiments, the terminal can obtain the effective frequency band portion and the target frequency band portion to be processed in the target audio through multi-layer wavelet transform. Thus, the terminal can process the audio signal corresponding to the target frequency band portion to achieve the effect of enabling hearing-impaired users to hear the complete audio signal.

[0066] In one embodiment, down-modulating a target portion of an audio signal to obtain a down-modulated target portion audio signal includes: for each target portion of the audio signal, obtaining the wavelet transform level of the target frequency band corresponding to the target portion of the audio signal; determining the corresponding down-modulation amplitude based on the wavelet transform level of the target portion of the audio signal, and down-modulating the target portion of the audio signal based on the down-modulation amplitude to obtain a down-modulated target portion audio signal; the down-modulation amplitude is negatively correlated with the wavelet transform level of the target portion of the audio signal.

[0067] In this embodiment, the wavelet transform can include multiple layers, each of which generates a corresponding target frequency band. The number of wavelet transform layers can be determined based on the effective frequency band and the cutoff frequency of the target audio. Therefore, there can be multiple target frequency bands to be processed, each corresponding to a wavelet transform layer and a target audio signal. For each target audio signal, the terminal can obtain the wavelet transform layer number of the target frequency band. Based on the wavelet transform layer number of the target audio signal, the terminal can determine the corresponding pitch reduction amplitude. The pitch reduction amplitude is negatively correlated with the wavelet transform layer number of the target audio signal; that is, the larger the layer number of the target audio signal, the smaller the pitch reduction amplitude. Thus, the terminal can reduce the pitch of the target audio signal based on the determined pitch reduction amplitude to obtain a pitch-reduced target audio signal.

[0068] Specifically, the results of the above multi-layer wavelet transform can be as follows: Figure 3 As shown, taking an example with M=2 wavelet transform layers, the terminal performs multi-layer wavelet transform on the target audio with a cutoff frequency of 16kHz using MATLAB. This yields the target frequency bands to be processed, including [4kHz, 8kHz] and [8kHz, 16kHz]. The terminal can then acquire the corresponding target audio signals for each of these frequency bands within the target audio. Furthermore, [the text abruptly ends here, likely due to an incomplete translation or missing information]. Figure 3It is known that the target frequency band within the range of [4kHz, 8kHz] corresponds to 2 layers, while the target frequency band within the range of [8kHz, 16kHz] corresponds to 1 layer. Therefore, the down-modulation of the target audio signal within the range of [8kHz, 16kHz] can be greater than that within the range of [4kHz, 8kHz]. For example, the terminal can determine the down-modulation of the target audio signal within the range of [8kHz, 16kHz] as down by two octaves, and the down-modulation of the target audio signal within the range of [4kHz, 8kHz] as down by one octave. For the audio portion corresponding to the valid frequency band, the terminal may not perform any down-modulation processing.

[0069] Through this embodiment, the terminal can determine the pitch reduction amplitude based on the number of wavelet transform layers, and perform corresponding pitch reduction processing on the target part of the audio signal based on the pitch reduction amplitude, so that the target part of the audio signal based on the pitch reduction can obtain audio suitable for the target user, and achieve the effect of enabling hearing-impaired users to hear the complete audio signal.

[0070] In one embodiment, down-pitching the target audio signal to obtain a down-pitch target audio signal includes: acquiring the percussion component and chord component in the target audio signal; performing a first down-pitch processing on the percussion component according to a waveform similarity superposition algorithm, and a second down-pitch processing on the chord component according to a variable speed constant pitch algorithm; and up-sampling the target audio signal that has undergone the first and second down-pitch processing to obtain the down-pitch target audio signal.

[0071] In this embodiment, the target audio signal may contain various components, such as percussion components and chord components. For different components, the terminal can use different pitch reduction methods. The specific pitch reduction value can be determined based on the wavelet transform level corresponding to the target audio signal. During pitch reduction, the terminal can acquire the percussion and chord components from the target audio signal. For the percussion components, the terminal can perform a first pitch reduction process using the wsola (Waveform Similarity Overlap Add) algorithm. For the chord components, the terminal can perform a second pitch reduction process using the Phase Vocoder algorithm. The wsola algorithm is a simple sound speed conversion method that achieves pitch reduction without changing speed by superimposing waveforms. The Phase Vocoder algorithm achieves pitch reduction without changing speed by preserving the original amplitude in the frequency domain and then calculating the phase of the next frame based on time, thereby predicting a new frame and achieving pitch reduction without changing speed.

[0072] After the terminal performs pitch reduction processing on the percussion component and chord component in the target audio signal through the first pitch reduction processing and the second pitch reduction processing respectively, it can upsample the processed target audio signal to obtain the pitch-reduced target audio signal.

[0073] Furthermore, the target audio signal obtained from the first and second pitch-down processing may have a mismatch in rhythm with the original target audio signal, which can be determined, for example, by signal length detection. Therefore, in one embodiment, the terminal can detect whether the signal length difference between the pitch-down target audio signal and the original target audio signal is greater than a preset threshold. If so, the terminal determines that the rhythm of the pitch-down target audio signal is mismatched with the original target audio signal. The terminal can then determine not to use the combination of the first and second pitch-down processing to down-pitch the target audio signal, but instead to perform a third pitch-down processing on the target audio signal according to the variable speed pitch-invariant algorithm. The terminal can then upsample the target audio signal after the third pitch-down processing to obtain the down-pitch target audio signal.

[0074] Specifically, during pitch reduction, the terminal can first reconstruct the audio from the effective frequency band portion and the target frequency band portion of the target audio, obtaining the audio signal corresponding to the effective frequency band portion and the target portion audio signal corresponding to the target frequency band portion. The terminal can use MATLAB's waverec function, a multi-scale wavelet reconstruction function, to obtain the effective mid-to-high frequency signals distributed in the corresponding frequency bands, for example... Figure 2The [4kHz, 8kHz] and [8kHz, 16kHz] frequencies are used. To compress the reconstructed target audio signal into a lower frequency band, i.e., within the aforementioned effective frequency range, such as [0Hz, 4kHz], the terminal can down-modulate the target audio signal in the [8kHz, 16kHz] band by 1 / 4 (two octaves) and down-modulate the target audio signal in the [4kHz, 8kHz] band by one octave.

[0075] The process of down-modulating the target audio signal at the terminal can be as follows: Figure 3 As shown, Figure 3 This is a flowchart illustrating the pitch reduction step in one embodiment. The terminal may preferentially use the HPS (harmonic percussion separation) scheme to reduce the pitch of the target audio signal. The HPS scheme includes using the aforementioned wsola to slow down the percussion components of the target audio signal, and using the aforementioned phase vocoder scheme to reduce the pitch of the harmonics. The terminal can then upsample the target audio signal processed by the aforementioned wsola and phase vocoder to obtain the final pitch-reduced target audio signal, such as pitch-reduced instrumental music.

[0076] It should be noted that the terminal can perform beat alignment detection on the down-pitch target audio signal. For example, the terminal can determine whether the beat is aligned by detecting the signal length. If the terminal detects that the signal length difference between the down-pitch target audio signal and the original down-pitch target audio signal is greater than a preset threshold, the terminal can adopt another down-pitch scheme, that is, instead of performing HPS processing, it uses a phase vocoder to slow down the target audio signal, upsamples the processed target audio signal, and obtains the down-pitch target audio signal, thus achieving beat alignment.

[0077] Through the above embodiments, the terminal can use different pitch reduction methods to reduce the pitch of different components in the target audio signal. Furthermore, if it detects that the beat is out of sync after pitch reduction, it can also employ alternative pitch reduction schemes to achieve beat alignment. Thus, the terminal can synthesize target audio suitable for the target user based on the pitch-reduced target audio signal, enabling hearing-impaired users to hear the complete audio signal.

[0078] In one embodiment, obtaining target audio suitable for a target user based on the synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the target portion audio signal with reduced pitch includes: obtaining the synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the target portion audio signal with reduced pitch; performing low-pass filtering on the synthesis result; and obtaining target audio suitable for the target user based on the low-pass filtered synthesis result.

[0079] In this embodiment, after down-modulating the target audio signal, the terminal can synthesize it with the audio signal corresponding to the effective frequency band of the target audio. The audio signal corresponding to the effective frequency band of the target audio can be obtained by the terminal through audio reconstruction of the effective frequency band of the target audio. Since there can be multiple target frequency bands to be processed, the down-modulated target audio signal can be multiple signals. The terminal can obtain the synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the down-modulated target audio signal to obtain the preliminarily processed target audio. The terminal can also perform low-pass filtering on the synthesis result and obtain the target audio suitable for the target user based on the low-pass filtered synthesis result.

[0080] Specifically, taking instrumental music as the target audio as an example, after obtaining multiple target audio signals through the aforementioned pitch reduction, the terminal can superimpose these signals with the low-frequency signals, that is, superimpose them with the audio signals corresponding to the effective frequency bands mentioned above, to obtain preliminary low-frequency instrumental music. To prevent image residue from appearing in the high-frequency signals after low-frequency reconstruction, the terminal can perform low-pass filtering on the superimposed low-frequency instrumental music. To obtain two cascaded fourth-order Butterworth filters (LR filters) with a cutoff frequency of 4kHz, the terminal, taking a signal system with a sampling rate of 48kHz as an example, can perform low-pass filtering on the superimposed low-frequency instrumental music as follows: Figure 4 As shown, Figure 4 This is a flowchart illustrating the low-pass filtering step in one embodiment. Amplitude-frequency curve 400 represents the audio signal after low-pass filtering based on the original Butterworth filter, and amplitude-frequency curve 402 represents the signal obtained after processing by the two cascaded fourth-order Butterworth filters, i.e., the final low-frequency pure music, thus allowing the terminal to obtain target audio suitable for the target user.

[0081] Through this embodiment, the terminal can obtain the target audio suitable for the target user by superimposing multiple partial audio signals and processing them through low-pass filtering, thereby enabling hearing-impaired users to hear the complete audio signal.

[0082] In one embodiment, such as Figure 5 As shown, Figure 5This is a flowchart illustrating the audio processing method in another embodiment. In this embodiment, taking instrumental music as the target audio as an example, the terminal first downsamples the input instrumental music to fs = 32kHz, obtaining the corresponding cutoff frequency fw = fs / 2 = 16kHz. The terminal can determine the wavelet transform level M based on the effective frequency band of the target user, taking the effective frequency band as [0kHz, 4kHz] as an example. If the terminal determines M = 2, then the terminal can perform M-order wavelet decomposition on the target audio to obtain a low-frequency signal and multiple high-frequency signals. The low-frequency signal can be the effective frequency band portion of the target audio, such as [0kHz, 4kHz], and the high-frequency signals can be the target audio signal to be processed, such as the frequency bands [4kHz, 8kHz] and [8kHz, 16kHz].

[0083] The terminal can perform audio reconstruction on low-frequency and high-frequency signals separately to obtain the audio signal corresponding to the effective frequency band and the target audio signal corresponding to the target frequency band. The terminal can then down-pitch the target audio signal and superimpose it with the audio signal corresponding to the effective frequency band. Finally, the terminal applies a low-pass filter to the superimposed audio signal to obtain the final low-frequency pure music, which is the target audio suitable for the aforementioned target user.

[0084] Through the above embodiments, the terminal reduces the high-frequency portion of the audio by combining it with the effective frequency bands that the target user can hear, thereby enabling hearing-impaired users to hear the complete audio signal. Furthermore, the effective frequency bands in this solution are selectable, allowing target users with different effective frequency bands to generate corresponding target audio, thus improving the applicability of the target audio.

[0085] It should be understood that although the steps in the flowcharts of the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0086] In one embodiment, a computer device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, NFC (Near Field Communication), or other technologies. When executed by the processor, the computer program implements an audio processing method. The display unit is used to form a visually visible image and can be a display screen, a projection device, or a virtual reality imaging device. The display screen can be an LCD screen or an e-ink display screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device casing, or an external keyboard, touchpad, or mouse.

[0087] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0088] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the audio processing method described above.

[0089] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the audio processing method described above.

[0090] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the audio processing method described above.

[0091] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0092] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0093] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0094] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An audio processing method, characterized in that, The method includes: Obtain the target audio; Identify the effective frequency bands for the target users; Based on the effective frequency band, a wavelet transform is performed on the downsampled target audio to obtain the effective frequency band portion and the target frequency band portion to be processed from the target audio. Acquiring the target portion audio signal corresponding to the target frequency band portion, and performing down-modulation processing on the target portion audio signal to obtain a down-modulated target portion audio signal, includes: for each target portion audio signal, down-modulating the target portion audio signal according to the down-modulation magnitude to obtain a down-modulated target portion audio signal; the down-modulation magnitude is determined according to the wavelet transform level of the target frequency band portion corresponding to the target portion audio signal; the down-modulation magnitude is negatively correlated with the wavelet transform level corresponding to the target portion audio signal. Based on the synthesis result of the audio signal corresponding to the effective frequency band portion of the target audio and the target portion audio signal with reduced pitch, a target audio suitable for the target user is obtained.

2. The method according to claim 1, characterized in that, The step of performing wavelet transform on the downsampled target audio based on the effective frequency band to obtain the effective frequency band portion and the target frequency band portion to be processed includes: Obtain the cutoff frequency corresponding to the frequency of the downsampled target audio; The corresponding wavelet transform layer number is determined based on the effective frequency band and the cutoff frequency; Using the cutoff frequency as the initial frequency, the target audio is subjected to wavelet transform according to the wavelet transform level and the initial frequency to obtain candidate low-frequency and candidate high-frequency components. The candidate low-frequency portion containing the frequency corresponding to the effective frequency band is used as the new initial frequency. The step of performing wavelet transform on the target audio according to the wavelet transform level and the initial frequency is returned until the wavelet transform level is reached. The effective frequency band portion of the target audio is obtained according to the candidate low-frequency portion with the smallest frequency range, and the target frequency band portion to be processed is obtained according to at least one candidate high-frequency portion.

3. The method according to claim 1, characterized in that, The step of acquiring the target portion audio signal corresponding to the target frequency band portion includes: The target frequency band portion is reconstructed using a multi-scale wavelet reconstruction function to obtain the target portion audio signal corresponding to the target frequency band portion.

4. The method according to any one of claims 1-2, characterized in that, The step of down-pitch processing the target portion of the audio signal to obtain the down-pitch target portion of the audio signal includes: Obtain the percussion components and chord components from the target portion of the audio signal; The percussion components are subjected to a first pitch reduction process based on a waveform similarity superposition algorithm, and the chord components are subjected to a second pitch reduction process based on a speed-variable pitch-invariant algorithm. The target portion of the audio signal that has undergone the first and second down-pitch processing is upsampled to obtain the down-pitch target portion of the audio signal.

5. The method according to claim 4, characterized in that, After performing a first pitch reduction processing on the percussion components according to the waveform similarity superposition algorithm, and a second pitch reduction processing on the chord components according to the speed-variable pitch-invariant algorithm, the method further includes: If the difference in signal length between the target portion of the audio signal to be down-pitched and the target portion of the audio signal before down-pitching is greater than a preset threshold, the target portion of the audio signal is subjected to a third down-pitching process according to the variable speed unchanged pitch algorithm. The target audio signal after the third down-pitch processing is upsampled to obtain the down-pitch target audio signal.

6. The method according to claim 1, characterized in that, The step of obtaining target audio suitable for the target user based on the synthesis result of the audio signal corresponding to the effective frequency band portion of the target audio and the down-pitch target portion audio signal includes: The synthesis result of the audio signal corresponding to the effective frequency band of the target audio and the target portion audio signal with reduced pitch is obtained; The synthesized result is low-pass filtered, and the target audio suitable for the target user is obtained based on the low-pass filtered synthesized result.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images